From mboxrd@z Thu Jan  1 00:00:00 1970
From: Ashok Anand <ashok.anand@gmail.com>
Subject: issue using SRIOV "Unable to start - perhaps the PF
 driver is not up yet", while PF driver is actually up
Date: Mon, 10 Sep 2012 15:54:34 +0530
Message-ID: <CALQsBOXFVM+cdhmke5JDJmns2XUpHjyDOVN6R5qrrY9nTOR3SQ@mail.gmail.com>
References: <CALQsBOWDE_KqpntG8WJo86omLP3+rhQ0LDMW5vB752ubpF1FAA@mail.gmail.com>
Mime-Version: 1.0
Content-Type: multipart/mixed; boundary="===============2699084713964537592=="
Return-path: <xen-devel-bounces@lists.xen.org>
In-Reply-To: <CALQsBOWDE_KqpntG8WJo86omLP3+rhQ0LDMW5vB752ubpF1FAA@mail.gmail.com>
List-Unsubscribe: <http://lists.xen.org/cgi-bin/mailman/options/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xen.org>
List-Help: <mailto:xen-devel-request@lists.xen.org?subject=help>
List-Subscribe: <http://lists.xen.org/cgi-bin/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=subscribe>
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: xen-devel@lists.xen.org
List-Id: xen-devel@lists.xenproject.org

--===============2699084713964537592==
Content-Type: multipart/alternative; boundary=f46d04426d22348a2104c9565da8

--f46d04426d22348a2104c9565da8
Content-Type: text/plain; charset=ISO-8859-1

Hi,

I am trying to use SRIOV, usnig xen 4.0 on debian.

On my dom0, xm pci-list-assignable-devices show that
0000:0f:10.0
0000:0f:10.2
0000:0f:10.4
0000:0f:10.6
0000:0f:11.2
0000:0f:11.4
0000:0f:11.6

these virtual interfaces correspond to eth2,

i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0

on my dom0 machine, i can ping other machine using eth2, (implying PF on
eth2 is active)

on my domU machine, when i attach the virtual function, i get the following
messages

[ 2282.688356] ixgbevf 0000:00:00.0: Xen PCI mapped GSI0 to IRQ28
[ 2282.688470] ixgbevf 0000:00:00.0: setting latency timer to 64
[ 2282.690187] ixgbevf 0000:00:00.0: PF still in reset state, assigning new
address

while PF on eth2 is  there and active, since i can ping other machine.

Now, when I try to bring up the VF interface on domU,
 i get the following error

 2476.295582] Unable to start - perhaps the PF Driver isn't up yet
SIOCSIFFLAGS: Network is down
[ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
SIOCSIFFLAGS: Network is down

and on dmesg on domU,
[ 2476.295582] Unable to start - perhaps the PF Driver isn't up yet
[ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet


Any thoughts on what could be wrong? I have been struggling with this for
quite some time
and would really appreciate your thoughts on it.

Thanks,
Ashok

--f46d04426d22348a2104c9565da8
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div class=3D"gmail_quote">Hi,=A0<div><br></div><div>I am trying to use SRI=
OV, usnig xen 4.0 on debian.=A0</div><div><br></div><div>On my dom0, xm pci=
-list-assignable-devices show that</div><div><div>0000:0f:10.0</div><div>00=
00:0f:10.2</div>
<div>0000:0f:10.4</div>
<div>0000:0f:10.6</div><div>0000:0f:11.2</div><div>0000:0f:11.4</div><div>0=
000:0f:11.6</div></div><div><br></div><div>these virtual interfaces corresp=
ond to eth2,=A0</div><div><br></div><div>i attach 0f:10.0 to a domU ubuntu =
machine,=A0xm pci-attach ubuntu 0f:10.0</div>

<div><br></div><div>on my dom0 machine, i can ping other machine using eth2=
, (implying PF on eth2 is active)</div><div><br></div><div>on my domU machi=
ne, when i attach the virtual function, i get the following messages</div>

<div><br></div><div><div><a href=3D"tel:%5B%202282.688356" value=3D"+122826=
88356" target=3D"_blank">[ 2282.688356</a>] ixgbevf 0000:00:00.0: Xen PCI m=
apped GSI0 to IRQ28</div><div><a href=3D"tel:%5B%202282.688470" value=3D"+1=
2282688470" target=3D"_blank">[ 2282.688470</a>] ixgbevf 0000:00:00.0: sett=
ing latency timer to 64</div>
<div><a href=3D"tel:%5B%202282.690187" value=3D"+12282690187" target=3D"_bl=
ank">[ 2282.690187</a>] ixgbevf 0000:00:00.0: PF still in reset state, assi=
gning new address</div>
</div><div><br></div><div>while PF on eth2 is =A0there and active, since i =
can ping other machine.</div><div><br></div><div>Now, when I try to bring u=
p the VF interface on domU,=A0</div><div>=A0i get the following error</div>=
<div>

<br></div><div><div>=A02476.295582] Unable to start - perhaps the PF Driver=
 isn&#39;t up yet</div><div>SIOCSIFFLAGS: Network is down</div><div>[ 2476.=
296917] Unable to start - perhaps the PF Driver isn&#39;t up yet</div><div>

SIOCSIFFLAGS: Network is down</div></div><div><br></div><div>and on dmesg o=
n domU,=A0</div><div><div>[ 2476.295582] Unable to start - perhaps the PF D=
river isn&#39;t up yet</div><div>[ 2476.296917] Unable to start - perhaps t=
he PF Driver isn&#39;t up yet</div>

</div><div><br></div><div><br></div><div>Any thoughts on what could be wron=
g? I have been struggling with this for quite some time</div><div>and would=
 really appreciate your thoughts on it.</div><div><br></div><div>Thanks,=A0=
</div>
<span class=3D"HOEnZb"><font color=3D"#888888">
<div>Ashok</div><div><br></div><div><br></div><div><br></div><div><br></div=
><div><br></div>
</font></span></div><br>

--f46d04426d22348a2104c9565da8--


--===============2699084713964537592==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

--===============2699084713964537592==--

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: issue using SRIOV "Unable to start - perhaps the PF
 driver is not up yet", while PF driver is actually up
Date: Mon, 10 Sep 2012 11:26:33 +0100
Message-ID: <504DC059.4070003@citrix.com>
References: <CALQsBOWDE_KqpntG8WJo86omLP3+rhQ0LDMW5vB752ubpF1FAA@mail.gmail.com>
	<CALQsBOXFVM+cdhmke5JDJmns2XUpHjyDOVN6R5qrrY9nTOR3SQ@mail.gmail.com>
Mime-Version: 1.0
Content-Type: multipart/mixed; boundary="===============6736997857128962086=="
Return-path: <xen-devel-bounces@lists.xen.org>
In-Reply-To: <CALQsBOXFVM+cdhmke5JDJmns2XUpHjyDOVN6R5qrrY9nTOR3SQ@mail.gmail.com>
List-Unsubscribe: <http://lists.xen.org/cgi-bin/mailman/options/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xen.org>
List-Help: <mailto:xen-devel-request@lists.xen.org?subject=help>
List-Subscribe: <http://lists.xen.org/cgi-bin/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=subscribe>
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: xen-devel@lists.xen.org
List-Id: xen-devel@lists.xenproject.org

--===============6736997857128962086==
Content-Type: multipart/alternative;
	boundary="------------070502040207030606070503"

--------------070502040207030606070503
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 7bit


On 10/09/12 11:24, Ashok Anand wrote:
> Hi, 
>
> I am trying to use SRIOV, usnig xen 4.0 on debian. 
>
> On my dom0, xm pci-list-assignable-devices show that
> 0000:0f:10.0
> 0000:0f:10.2
> 0000:0f:10.4
> 0000:0f:10.6
> 0000:0f:11.2
> 0000:0f:11.4
> 0000:0f:11.6
>
> these virtual interfaces correspond to eth2,

Are you sure?  0f:10.0 is probably the physical function which you must
keep in dom0.

~Andrew

>
> i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0
>
> on my dom0 machine, i can ping other machine using eth2, (implying PF
> on eth2 is active)
>
> on my domU machine, when i attach the virtual function, i get the
> following messages
>
> [ 2282.688356 <tel:%5B%202282.688356>] ixgbevf 0000:00:00.0: Xen PCI
> mapped GSI0 to IRQ28
> [ 2282.688470 <tel:%5B%202282.688470>] ixgbevf 0000:00:00.0: setting
> latency timer to 64
> [ 2282.690187 <tel:%5B%202282.690187>] ixgbevf 0000:00:00.0: PF still
> in reset state, assigning new address
>
> while PF on eth2 is  there and active, since i can ping other machine.
>
> Now, when I try to bring up the VF interface on domU, 
>  i get the following error
>
>  2476.295582] Unable to start - perhaps the PF Driver isn't up yet
> SIOCSIFFLAGS: Network is down
> [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
> SIOCSIFFLAGS: Network is down
>
> and on dmesg on domU, 
> [ 2476.295582] Unable to start - perhaps the PF Driver isn't up yet
> [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
>
>
> Any thoughts on what could be wrong? I have been struggling with this
> for quite some time
> and would really appreciate your thoughts on it.
>
> Thanks, 
> Ashok
>
>
>
>
>
>

-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com


--------------070502040207030606070503
Content-Type: text/html; charset="ISO-8859-1"
Content-Transfer-Encoding: 7bit

<html>
  <head>
    <meta content="text/html; charset=ISO-8859-1"
      http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <br>
    <div class="moz-cite-prefix">On 10/09/12 11:24, Ashok Anand wrote:<br>
    </div>
    <blockquote
cite="mid:CALQsBOXFVM+cdhmke5JDJmns2XUpHjyDOVN6R5qrrY9nTOR3SQ@mail.gmail.com"
      type="cite">
      <div class="gmail_quote">Hi,&nbsp;
        <div><br>
        </div>
        <div>I am trying to use SRIOV, usnig xen 4.0 on debian.&nbsp;</div>
        <div><br>
        </div>
        <div>On my dom0, xm pci-list-assignable-devices show that</div>
        <div>
          <div>0000:0f:10.0</div>
          <div>0000:0f:10.2</div>
          <div>0000:0f:10.4</div>
          <div>0000:0f:10.6</div>
          <div>0000:0f:11.2</div>
          <div>0000:0f:11.4</div>
          <div>0000:0f:11.6</div>
        </div>
        <div><br>
        </div>
        <div>these virtual interfaces correspond to eth2, <br>
        </div>
      </div>
    </blockquote>
    <br>
    Are you sure?&nbsp; 0f:10.0 is probably the physical function which you
    must keep in dom0.<br>
    <br>
    ~Andrew<br>
    <br>
    <blockquote
cite="mid:CALQsBOXFVM+cdhmke5JDJmns2XUpHjyDOVN6R5qrrY9nTOR3SQ@mail.gmail.com"
      type="cite">
      <div class="gmail_quote">
        <div><br>
        </div>
        <div>i attach 0f:10.0 to a domU ubuntu machine,&nbsp;xm pci-attach
          ubuntu 0f:10.0</div>
        <div><br>
        </div>
        <div>on my dom0 machine, i can ping other machine using eth2,
          (implying PF on eth2 is active)</div>
        <div><br>
        </div>
        <div>on my domU machine, when i attach the virtual function, i
          get the following messages</div>
        <div><br>
        </div>
        <div>
          <div><a moz-do-not-send="true" href="tel:%5B%202282.688356"
              value="+12282688356" target="_blank">[ 2282.688356</a>]
            ixgbevf 0000:00:00.0: Xen PCI mapped GSI0 to IRQ28</div>
          <div><a moz-do-not-send="true" href="tel:%5B%202282.688470"
              value="+12282688470" target="_blank">[ 2282.688470</a>]
            ixgbevf 0000:00:00.0: setting latency timer to 64</div>
          <div><a moz-do-not-send="true" href="tel:%5B%202282.690187"
              value="+12282690187" target="_blank">[ 2282.690187</a>]
            ixgbevf 0000:00:00.0: PF still in reset state, assigning new
            address</div>
        </div>
        <div><br>
        </div>
        <div>while PF on eth2 is &nbsp;there and active, since i can ping
          other machine.</div>
        <div><br>
        </div>
        <div>Now, when I try to bring up the VF interface on domU,&nbsp;</div>
        <div>&nbsp;i get the following error</div>
        <div>
          <br>
        </div>
        <div>
          <div>&nbsp;2476.295582] Unable to start - perhaps the PF Driver
            isn't up yet</div>
          <div>SIOCSIFFLAGS: Network is down</div>
          <div>[ 2476.296917] Unable to start - perhaps the PF Driver
            isn't up yet</div>
          <div>
            SIOCSIFFLAGS: Network is down</div>
        </div>
        <div><br>
        </div>
        <div>and on dmesg on domU,&nbsp;</div>
        <div>
          <div>[ 2476.295582] Unable to start - perhaps the PF Driver
            isn't up yet</div>
          <div>[ 2476.296917] Unable to start - perhaps the PF Driver
            isn't up yet</div>
        </div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div>Any thoughts on what could be wrong? I have been struggling
          with this for quite some time</div>
        <div>and would really appreciate your thoughts on it.</div>
        <div><br>
        </div>
        <div>Thanks,&nbsp;</div>
        <span class="HOEnZb"><font color="#888888">
            <div>Ashok</div>
            <div><br>
            </div>
            <div><br>
            </div>
            <div><br>
            </div>
            <div><br>
            </div>
            <div><br>
            </div>
          </font></span></div>
      <br>
    </blockquote>
    <br>
    <pre class="moz-signature" cols="72">-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, <a class="moz-txt-link-freetext" href="http://www.citrix.com">http://www.citrix.com</a></pre>
  </body>
</html>

--------------070502040207030606070503--


--===============6736997857128962086==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

--===============6736997857128962086==--

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Pasi =?iso-8859-1?Q?K=E4rkk=E4inen?= <pasik@iki.fi>
Subject: Re: issue using SRIOV "Unable to start - perhaps the PF
 driver is not up yet", while PF driver is actually up
Date: Mon, 10 Sep 2012 13:27:37 +0300
Message-ID: <20120910102737.GG8912@reaktio.net>
References: <CALQsBOWDE_KqpntG8WJo86omLP3+rhQ0LDMW5vB752ubpF1FAA@mail.gmail.com>
	<CALQsBOXFVM+cdhmke5JDJmns2XUpHjyDOVN6R5qrrY9nTOR3SQ@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: <xen-devel-bounces@lists.xen.org>
Content-Disposition: inline
In-Reply-To: <CALQsBOXFVM+cdhmke5JDJmns2XUpHjyDOVN6R5qrrY9nTOR3SQ@mail.gmail.com>
List-Unsubscribe: <http://lists.xen.org/cgi-bin/mailman/options/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xen.org>
List-Help: <mailto:xen-devel-request@lists.xen.org?subject=help>
List-Subscribe: <http://lists.xen.org/cgi-bin/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=subscribe>
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Ashok Anand <ashok.anand@gmail.com>
Cc: xen-devel@lists.xen.org
List-Id: xen-devel@lists.xenproject.org

On Mon, Sep 10, 2012 at 03:54:34PM +0530, Ashok Anand wrote:
>    Hi,
>    I am trying to use SRIOV, usnig xen 4.0 on debian.
>    On my dom0, xm pci-list-assignable-devices show that
>    0000:0f:10.0
>    0000:0f:10.2
>    0000:0f:10.4
>    0000:0f:10.6
>    0000:0f:11.2
>    0000:0f:11.4
>    0000:0f:11.6
>    these virtual interfaces correspond to eth2,
>    i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0
>    on my dom0 machine, i can ping other machine using eth2, (implying PF on
>    eth2 is active)
>    on my domU machine, when i attach the virtual function, i get the
>    following messages
>    [1][ 2282.688356] ixgbevf 0000:00:00.0: Xen PCI mapped GSI0 to IRQ28
>    [2][ 2282.688470] ixgbevf 0000:00:00.0: setting latency timer to 64
>    [3][ 2282.690187] ixgbevf 0000:00:00.0: PF still in reset state, assigning
>    new address
>    while PF on eth2 is  there and active, since i can ping other machine.
>    Now, when I try to bring up the VF interface on domU,
>     i get the following error
>     2476.295582] Unable to start - perhaps the PF Driver isn't up yet
>    SIOCSIFFLAGS: Network is down
>    [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
>    SIOCSIFFLAGS: Network is down
>    and on dmesg on domU,
>    [ 2476.295582] Unable to start - perhaps the PF Driver isn't up yet
>    [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
>    Any thoughts on what could be wrong? I have been struggling with this for
>    quite some time
>    and would really appreciate your thoughts on it.
>

Did you do "ifconfig ethX up" in dom0? 

-- Pasi

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Ashok Anand <ashok.anand@gmail.com>
Subject: Re: issue using SRIOV "Unable to start - perhaps the PF
 driver is not up yet", while PF driver is actually up
Date: Mon, 10 Sep 2012 16:04:01 +0530
Message-ID: <CALQsBOV0EsUY+eYWqtL-FVuTOWOk_MLPnC2CYYPhNU9t=sxt9g@mail.gmail.com>
References: <CALQsBOWDE_KqpntG8WJo86omLP3+rhQ0LDMW5vB752ubpF1FAA@mail.gmail.com>
	<CALQsBOXFVM+cdhmke5JDJmns2XUpHjyDOVN6R5qrrY9nTOR3SQ@mail.gmail.com>
	<504DC059.4070003@citrix.com>
Mime-Version: 1.0
Content-Type: multipart/mixed; boundary="===============3633803858675574930=="
Return-path: <xen-devel-bounces@lists.xen.org>
In-Reply-To: <504DC059.4070003@citrix.com>
List-Unsubscribe: <http://lists.xen.org/cgi-bin/mailman/options/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xen.org>
List-Help: <mailto:xen-devel-request@lists.xen.org?subject=help>
List-Subscribe: <http://lists.xen.org/cgi-bin/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=subscribe>
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: xen-devel@lists.xen.org
List-Id: xen-devel@lists.xenproject.org

--===============3633803858675574930==
Content-Type: multipart/alternative; boundary=e89a8fb1efa4f7ac5604c9567e47

--e89a8fb1efa4f7ac5604c9567e47
Content-Type: text/plain; charset=ISO-8859-1

Are you sure?  0f:10.0 is probably the physical function which you must
> keep in dom0.
>
> ~Andrew
>

I find from xm pci-list-assignable-devices  that 0f:10.0 is available - it
is also hidden from dom0 (using pciback.hide in kernel arguments)

if that is not the case, which physical function i can assign to domU?
i tried with 0f:11.6, and others too, but i got similar messages.


>
>
>  i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0
>
>  on my dom0 machine, i can ping other machine using eth2, (implying PF on
> eth2 is active)
>
>  on my domU machine, when i attach the virtual function, i get the
> following messages
>
>  [ 2282.688356] ixgbevf 0000:00:00.0: Xen PCI mapped GSI0 to IRQ28
> [ 2282.688470] ixgbevf 0000:00:00.0: setting latency timer to 64
> [ 2282.690187] ixgbevf 0000:00:00.0: PF still in reset state, assigning
> new address
>
>  while PF on eth2 is  there and active, since i can ping other machine.
>
>  Now, when I try to bring up the VF interface on domU,
>  i get the following error
>
>   2476.295582] Unable to start - perhaps the PF Driver isn't up yet
> SIOCSIFFLAGS: Network is down
> [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
>  SIOCSIFFLAGS: Network is down
>
>  and on dmesg on domU,
>  [ 2476.295582] Unable to start - perhaps the PF Driver isn't up yet
> [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
>
>
>  Any thoughts on what could be wrong? I have been struggling with this
> for quite some time
> and would really appreciate your thoughts on it.
>
>  Thanks,
>  Ashok
>
>
>
>
>
>
>
> --
> Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
> T: +44 (0)1223 225 900, http://www.citrix.com
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>
>

--e89a8fb1efa4f7ac5604c9567e47
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<br><br><div class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=
=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgc=
olor=3D"#FFFFFF" text=3D"#000000">
    Are you sure?=A0 0f:10.0 is probably the physical function which you
    must keep in dom0.<br>
    <br>
    ~Andrew</div></blockquote><div><br></div><div>I find from xm pci-list-a=
ssignable-devices =A0that 0f:10.0 is available - it is also hidden from dom=
0 (using pciback.hide in kernel arguments)</div><div><br></div><div>if that=
 is not the case, which physical function i can assign to domU?</div>
<div>i tried with 0f:11.6, and others too, but i got similar messages.</div=
><div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8e=
x;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor=3D"#FFFFFF" tex=
t=3D"#000000">
<div class=3D"im"><br>
    <br>
    <blockquote type=3D"cite">
      <div class=3D"gmail_quote">
        <div><br>
        </div>
        <div>i attach 0f:10.0 to a domU ubuntu machine,=A0xm pci-attach
          ubuntu 0f:10.0</div>
        <div><br>
        </div>
        <div>on my dom0 machine, i can ping other machine using eth2,
          (implying PF on eth2 is active)</div>
        <div><br>
        </div>
        <div>on my domU machine, when i attach the virtual function, i
          get the following messages</div>
        <div><br>
        </div>
        <div>
          <div><a href=3D"tel:%5B%202282.688356" value=3D"+12282688356" tar=
get=3D"_blank">[ 2282.688356</a>]
            ixgbevf 0000:00:00.0: Xen PCI mapped GSI0 to IRQ28</div>
          <div><a href=3D"tel:%5B%202282.688470" value=3D"+12282688470" tar=
get=3D"_blank">[ 2282.688470</a>]
            ixgbevf 0000:00:00.0: setting latency timer to 64</div>
          <div><a href=3D"tel:%5B%202282.690187" value=3D"+12282690187" tar=
get=3D"_blank">[ 2282.690187</a>]
            ixgbevf 0000:00:00.0: PF still in reset state, assigning new
            address</div>
        </div>
        <div><br>
        </div>
        <div>while PF on eth2 is =A0there and active, since i can ping
          other machine.</div>
        <div><br>
        </div>
        <div>Now, when I try to bring up the VF interface on domU,=A0</div>
        <div>=A0i get the following error</div>
        <div>
          <br>
        </div>
        <div>
          <div>=A02476.295582] Unable to start - perhaps the PF Driver
            isn&#39;t up yet</div>
          <div>SIOCSIFFLAGS: Network is down</div>
          <div>[ 2476.296917] Unable to start - perhaps the PF Driver
            isn&#39;t up yet</div>
          <div>
            SIOCSIFFLAGS: Network is down</div>
        </div>
        <div><br>
        </div>
        <div>and on dmesg on domU,=A0</div>
        <div>
          <div>[ 2476.295582] Unable to start - perhaps the PF Driver
            isn&#39;t up yet</div>
          <div>[ 2476.296917] Unable to start - perhaps the PF Driver
            isn&#39;t up yet</div>
        </div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div>Any thoughts on what could be wrong? I have been struggling
          with this for quite some time</div>
        <div>and would really appreciate your thoughts on it.</div>
        <div><br>
        </div>
        <div>Thanks,=A0</div>
        <span><font color=3D"#888888">
            <div>Ashok</div>
            <div><br>
            </div>
            <div><br>
            </div>
            <div><br>
            </div>
            <div><br>
            </div>
            <div><br>
            </div>
          </font></span></div>
      <br>
    </blockquote>
    <br>
    </div><span class=3D"HOEnZb"><font color=3D"#888888"><pre cols=3D"72">-=
-=20
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: <a href=3D"tel:%2B44%20%280%291223%20225%20900" value=3D"+441223225900" =
target=3D"_blank">+44 (0)1223 225 900</a>, <a href=3D"http://www.citrix.com=
" target=3D"_blank">http://www.citrix.com</a></pre>
  </font></span></div>

<br>_______________________________________________<br>
Xen-devel mailing list<br>
<a href=3D"mailto:Xen-devel@lists.xen.org">Xen-devel@lists.xen.org</a><br>
<a href=3D"http://lists.xen.org/xen-devel" target=3D"_blank">http://lists.x=
en.org/xen-devel</a><br>
<br></blockquote></div><br>

--e89a8fb1efa4f7ac5604c9567e47--


--===============3633803858675574930==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

--===============3633803858675574930==--

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: issue using SRIOV "Unable to start - perhaps the PF
 driver is not up yet", while PF driver is actually up
Date: Mon, 10 Sep 2012 11:38:44 +0100
Message-ID: <504DC334.6060201@citrix.com>
References: <CALQsBOWDE_KqpntG8WJo86omLP3+rhQ0LDMW5vB752ubpF1FAA@mail.gmail.com>
	<CALQsBOXFVM+cdhmke5JDJmns2XUpHjyDOVN6R5qrrY9nTOR3SQ@mail.gmail.com>
	<504DC059.4070003@citrix.com>
	<CALQsBOV0EsUY+eYWqtL-FVuTOWOk_MLPnC2CYYPhNU9t=sxt9g@mail.gmail.com>
Mime-Version: 1.0
Content-Type: multipart/mixed; boundary="===============7699979477034878547=="
Return-path: <xen-devel-bounces@lists.xen.org>
In-Reply-To: <CALQsBOV0EsUY+eYWqtL-FVuTOWOk_MLPnC2CYYPhNU9t=sxt9g@mail.gmail.com>
List-Unsubscribe: <http://lists.xen.org/cgi-bin/mailman/options/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xen.org>
List-Help: <mailto:xen-devel-request@lists.xen.org?subject=help>
List-Subscribe: <http://lists.xen.org/cgi-bin/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=subscribe>
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Ashok Anand <ashok.anand@gmail.com>
Cc: "xen-devel@lists.xen.org" <xen-devel@lists.xen.org>
List-Id: xen-devel@lists.xenproject.org

--===============7699979477034878547==
Content-Type: multipart/alternative;
	boundary="------------090908030301000108080305"

--------------090908030301000108080305
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 7bit

On 10/09/12 11:34, Ashok Anand wrote:
>
>
>     Are you sure?  0f:10.0 is probably the physical function which you
>     must keep in dom0.
>
>     ~Andrew
>
>
> I find from xm pci-list-assignable-devices  that 0f:10.0 is available
> - it is also hidden from dom0 (using pciback.hide in kernel arguments)
>
> if that is not the case, which physical function i can assign to domU?
> i tried with 0f:11.6, and others too, but i got similar messages.

What does lspci -vv for the physical function say?

As for the physical function: it is unsafe to pass physical functions to
a non-trusted guest, as the physical function has complete control over
the all the virtual functions, even if they are passed through to other
guests.  For that reason, the physical function should remain in dom0
(or a device driver domain if you are going for disaggregation).

>
>
>
>>
>>     i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu
>>     0f:10.0
>>
>>     on my dom0 machine, i can ping other machine using eth2,
>>     (implying PF on eth2 is active)
>>
>>     on my domU machine, when i attach the virtual function, i get the
>>     following messages
>>
>>     [ 2282.688356 <tel:%5B%202282.688356>] ixgbevf 0000:00:00.0: Xen
>>     PCI mapped GSI0 to IRQ28
>>     [ 2282.688470 <tel:%5B%202282.688470>] ixgbevf 0000:00:00.0:
>>     setting latency timer to 64
>>     [ 2282.690187 <tel:%5B%202282.690187>] ixgbevf 0000:00:00.0: PF
>>     still in reset state, assigning new address
>>
>>     while PF on eth2 is  there and active, since i can ping other
>>     machine.
>>
>>     Now, when I try to bring up the VF interface on domU, 
>>      i get the following error
>>
>>      2476.295582] Unable to start - perhaps the PF Driver isn't up yet
>>     SIOCSIFFLAGS: Network is down
>>     [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
>>     SIOCSIFFLAGS: Network is down
>>
>>     and on dmesg on domU, 
>>     [ 2476.295582] Unable to start - perhaps the PF Driver isn't up yet
>>     [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
>>
>>
>>     Any thoughts on what could be wrong? I have been struggling with
>>     this for quite some time
>>     and would really appreciate your thoughts on it.
>>
>>     Thanks, 
>>     Ashok
>>
>>
>>
>>
>>
>>
>
>     -- 
>     Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>     T: +44 (0)1223 225 900 <tel:%2B44%20%280%291223%20225%20900>, http://www.citrix.com
>
>
>     _______________________________________________
>     Xen-devel mailing list
>     Xen-devel@lists.xen.org <mailto:Xen-devel@lists.xen.org>
>     http://lists.xen.org/xen-devel
>
>

-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com


--------------090908030301000108080305
Content-Type: text/html; charset="ISO-8859-1"
Content-Transfer-Encoding: 7bit

<html>
  <head>
    <meta content="text/html; charset=ISO-8859-1"
      http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    On 10/09/12 11:34, Ashok Anand wrote:<br>
    <blockquote
cite="mid:CALQsBOV0EsUY+eYWqtL-FVuTOWOk_MLPnC2CYYPhNU9t=sxt9g@mail.gmail.com"
      type="cite"><br>
      <br>
      <div class="gmail_quote">
        <blockquote class="gmail_quote" style="margin:0 0 0
          .8ex;border-left:1px #ccc solid;padding-left:1ex">
          <div bgcolor="#FFFFFF" text="#000000"> Are you sure?&nbsp; 0f:10.0
            is probably the physical function which you must keep in
            dom0.<br>
            <br>
            ~Andrew</div>
        </blockquote>
        <div><br>
        </div>
        <div>I find from xm pci-list-assignable-devices &nbsp;that 0f:10.0 is
          available - it is also hidden from dom0 (using pciback.hide in
          kernel arguments)</div>
        <div><br>
        </div>
        <div>if that is not the case, which physical function i can
          assign to domU?</div>
        <div>i tried with 0f:11.6, and others too, but i got similar
          messages.</div>
      </div>
    </blockquote>
    <br>
    What does lspci -vv for the physical function say?<br>
    <br>
    As for the physical function: it is unsafe to pass physical
    functions to a non-trusted guest, as the physical function has
    complete control over the all the virtual functions, even if they
    are passed through to other guests.&nbsp; For that reason, the physical
    function should remain in dom0 (or a device driver domain if you are
    going for disaggregation).<br>
    <br>
    <blockquote
cite="mid:CALQsBOV0EsUY+eYWqtL-FVuTOWOk_MLPnC2CYYPhNU9t=sxt9g@mail.gmail.com"
      type="cite">
      <div class="gmail_quote">
        <div><br>
        </div>
        <blockquote class="gmail_quote" style="margin:0 0 0
          .8ex;border-left:1px #ccc solid;padding-left:1ex">
          <div bgcolor="#FFFFFF" text="#000000">
            <div class="im"><br>
              <br>
              <blockquote type="cite">
                <div class="gmail_quote">
                  <div><br>
                  </div>
                  <div>i attach 0f:10.0 to a domU ubuntu machine,&nbsp;xm
                    pci-attach ubuntu 0f:10.0</div>
                  <div><br>
                  </div>
                  <div>on my dom0 machine, i can ping other machine
                    using eth2, (implying PF on eth2 is active)</div>
                  <div><br>
                  </div>
                  <div>on my domU machine, when i attach the virtual
                    function, i get the following messages</div>
                  <div><br>
                  </div>
                  <div>
                    <div><a moz-do-not-send="true"
                        href="tel:%5B%202282.688356"
                        value="+12282688356" target="_blank">[
                        2282.688356</a>] ixgbevf 0000:00:00.0: Xen PCI
                      mapped GSI0 to IRQ28</div>
                    <div><a moz-do-not-send="true"
                        href="tel:%5B%202282.688470"
                        value="+12282688470" target="_blank">[
                        2282.688470</a>] ixgbevf 0000:00:00.0: setting
                      latency timer to 64</div>
                    <div><a moz-do-not-send="true"
                        href="tel:%5B%202282.690187"
                        value="+12282690187" target="_blank">[
                        2282.690187</a>] ixgbevf 0000:00:00.0: PF still
                      in reset state, assigning new address</div>
                  </div>
                  <div><br>
                  </div>
                  <div>while PF on eth2 is &nbsp;there and active, since i
                    can ping other machine.</div>
                  <div><br>
                  </div>
                  <div>Now, when I try to bring up the VF interface on
                    domU,&nbsp;</div>
                  <div>&nbsp;i get the following error</div>
                  <div> <br>
                  </div>
                  <div>
                    <div>&nbsp;2476.295582] Unable to start - perhaps the PF
                      Driver isn't up yet</div>
                    <div>SIOCSIFFLAGS: Network is down</div>
                    <div>[ 2476.296917] Unable to start - perhaps the PF
                      Driver isn't up yet</div>
                    <div> SIOCSIFFLAGS: Network is down</div>
                  </div>
                  <div><br>
                  </div>
                  <div>and on dmesg on domU,&nbsp;</div>
                  <div>
                    <div>[ 2476.295582] Unable to start - perhaps the PF
                      Driver isn't up yet</div>
                    <div>[ 2476.296917] Unable to start - perhaps the PF
                      Driver isn't up yet</div>
                  </div>
                  <div><br>
                  </div>
                  <div><br>
                  </div>
                  <div>Any thoughts on what could be wrong? I have been
                    struggling with this for quite some time</div>
                  <div>and would really appreciate your thoughts on it.</div>
                  <div><br>
                  </div>
                  <div>Thanks,&nbsp;</div>
                  <span><font color="#888888">
                      <div>Ashok</div>
                      <div><br>
                      </div>
                      <div><br>
                      </div>
                      <div><br>
                      </div>
                      <div><br>
                      </div>
                      <div><br>
                      </div>
                    </font></span></div>
                <br>
              </blockquote>
              <br>
            </div>
            <span class="HOEnZb"><font color="#888888">
                <pre cols="72">-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: <a moz-do-not-send="true" href="tel:%2B44%20%280%291223%20225%20900" value="+441223225900" target="_blank">+44 (0)1223 225 900</a>, <a moz-do-not-send="true" href="http://www.citrix.com" target="_blank">http://www.citrix.com</a></pre>
              </font></span></div>
          <br>
          _______________________________________________<br>
          Xen-devel mailing list<br>
          <a moz-do-not-send="true"
            href="mailto:Xen-devel@lists.xen.org">Xen-devel@lists.xen.org</a><br>
          <a moz-do-not-send="true"
            href="http://lists.xen.org/xen-devel" target="_blank">http://lists.xen.org/xen-devel</a><br>
          <br>
        </blockquote>
      </div>
      <br>
    </blockquote>
    <br>
    <pre class="moz-signature" cols="72">-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, <a class="moz-txt-link-freetext" href="http://www.citrix.com">http://www.citrix.com</a></pre>
  </body>
</html>

--------------090908030301000108080305--


--===============7699979477034878547==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

--===============7699979477034878547==--

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Ashok Anand <ashok.anand@gmail.com>
Subject: Re: issue using SRIOV "Unable to start - perhaps the PF
 driver is not up yet", while PF driver is actually up
Date: Mon, 10 Sep 2012 16:16:47 +0530
Message-ID: <CALQsBOXqT39-ZkOuTT-c58qUW6dihuXWfUi08YzjmgJzaW9YAA@mail.gmail.com>
References: <CALQsBOWDE_KqpntG8WJo86omLP3+rhQ0LDMW5vB752ubpF1FAA@mail.gmail.com>
	<CALQsBOXFVM+cdhmke5JDJmns2XUpHjyDOVN6R5qrrY9nTOR3SQ@mail.gmail.com>
	<504DC059.4070003@citrix.com>
	<CALQsBOV0EsUY+eYWqtL-FVuTOWOk_MLPnC2CYYPhNU9t=sxt9g@mail.gmail.com>
	<504DC334.6060201@citrix.com>
Mime-Version: 1.0
Content-Type: multipart/mixed; boundary="===============3163611272199496851=="
Return-path: <xen-devel-bounces@lists.xen.org>
In-Reply-To: <504DC334.6060201@citrix.com>
List-Unsubscribe: <http://lists.xen.org/cgi-bin/mailman/options/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xen.org>
List-Help: <mailto:xen-devel-request@lists.xen.org?subject=help>
List-Subscribe: <http://lists.xen.org/cgi-bin/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=subscribe>
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: "xen-devel@lists.xen.org" <xen-devel@lists.xen.org>
List-Id: xen-devel@lists.xenproject.org

--===============3163611272199496851==
Content-Type: multipart/alternative; boundary=e89a8fb1ffc2a69dac04c956ac5c

--e89a8fb1ffc2a69dac04c956ac5c
Content-Type: text/plain; charset=ISO-8859-1

I am passing virtual functions. please see below ,based on the message from
lspci.

> What does lspci -vv for the physical function say?
>
> As for the physical function: it is unsafe to pass physical functions to a
> non-trusted guest, as the physical function has complete control over the
> all the virtual functions, even if they are passed through to other
> guests.  For that reason, the physical function should remain in dom0 (or a
> device driver domain if you are going for disaggregation).
>

here is the message lspci -vv | grep Eth gives me,

0f:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
Network Connection (rev 01)
Subsystem: Intel Corporation Ethernet Server Adapter X520-2
0f:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
Network Connection (rev 01)
Subsystem: Intel Corporation Ethernet Server Adapter X520-2
0f:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
0f:10.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
0f:10.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
0f:10.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
0f:11.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
0f:11.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
0f:11.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
0f:11.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)

Here, 0f:00.0 is the physical function, while 0f:10.0 and others are
virtual functions -- so I am attaching virtual functions.




>
>
>
>
>>
>>
>>  i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0
>>
>>  on my dom0 machine, i can ping other machine using eth2, (implying PF
>> on eth2 is active)
>>
>>  on my domU machine, when i attach the virtual function, i get the
>> following messages
>>
>>  [ 2282.688356 <%5B%202282.688356>] ixgbevf 0000:00:00.0: Xen PCI mapped
>> GSI0 to IRQ28
>> [ 2282.688470 <%5B%202282.688470>] ixgbevf 0000:00:00.0: setting latency
>> timer to 64
>> [ 2282.690187 <%5B%202282.690187>] ixgbevf 0000:00:00.0: PF still in
>> reset state, assigning new address
>>
>>  while PF on eth2 is  there and active, since i can ping other machine.
>>
>>  Now, when I try to bring up the VF interface on domU,
>>  i get the following error
>>
>>   2476.295582] Unable to start - perhaps the PF Driver isn't up yet
>> SIOCSIFFLAGS: Network is down
>> [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
>>  SIOCSIFFLAGS: Network is down
>>
>>  and on dmesg on domU,
>>  [ 2476.295582] Unable to start - perhaps the PF Driver isn't up yet
>> [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
>>
>>
>>  Any thoughts on what could be wrong? I have been struggling with this
>> for quite some time
>> and would really appreciate your thoughts on it.
>>
>>  Thanks,
>>  Ashok
>>
>>
>>
>>
>>
>>
>>
>>  --
>> Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>> T: +44 (0)1223 225 900, http://www.citrix.com
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> http://lists.xen.org/xen-devel
>>
>>
>
> --
> Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
> T: +44 (0)1223 225 900, http://www.citrix.com
>
>

--e89a8fb1ffc2a69dac04c956ac5c
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div class=3D"gmail_quote"><div><br></div><div>I am passing virtual functio=
ns. please see below ,based on the message from lspci.=A0</div><blockquote =
class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid=
;padding-left:1ex">
<div bgcolor=3D"#FFFFFF" text=3D"#000000">What does lspci -vv for the physi=
cal function say?<br>
    <br>
    As for the physical function: it is unsafe to pass physical
    functions to a non-trusted guest, as the physical function has
    complete control over the all the virtual functions, even if they
    are passed through to other guests.=A0 For that reason, the physical
    function should remain in dom0 (or a device driver domain if you are
    going for disaggregation).</div></blockquote><div><br></div><div>here i=
s the message lspci -vv | grep Eth gives me,=A0</div><div><br></div><div><d=
iv>0f:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SF=
P+ Network Connection (rev 01)</div>
<div><span class=3D"Apple-tab-span" style=3D"white-space:pre">	</span>Subsy=
stem: Intel Corporation Ethernet Server Adapter X520-2</div><div>0f:00.1 Et=
hernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Co=
nnection (rev 01)</div>
<div><span class=3D"Apple-tab-span" style=3D"white-space:pre">	</span>Subsy=
stem: Intel Corporation Ethernet Server Adapter X520-2</div><div>0f:10.0 Et=
hernet controller: Intel Corporation 82599 Ethernet Controller Virtual Func=
tion (rev 01)</div>
<div>0f:10.2 Ethernet controller: Intel Corporation 82599 Ethernet Controll=
er Virtual Function (rev 01)</div><div>0f:10.4 Ethernet controller: Intel C=
orporation 82599 Ethernet Controller Virtual Function (rev 01)</div><div>
0f:10.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller Vi=
rtual Function (rev 01)</div><div>0f:11.0 Ethernet controller: Intel Corpor=
ation 82599 Ethernet Controller Virtual Function (rev 01)</div><div>0f:11.2=
 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual F=
unction (rev 01)</div>
<div>0f:11.4 Ethernet controller: Intel Corporation 82599 Ethernet Controll=
er Virtual Function (rev 01)</div><div>0f:11.6 Ethernet controller: Intel C=
orporation 82599 Ethernet Controller Virtual Function (rev 01)</div></div>
<div><br></div><div>Here, 0f:00.0 is the physical function, while 0f:10.0 a=
nd others are virtual functions -- so I am attaching virtual functions.=A0<=
/div><div><br></div><div><br></div><div>=A0</div><blockquote class=3D"gmail=
_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:=
1ex">
<div bgcolor=3D"#FFFFFF" text=3D"#000000"><div><div class=3D"h5"><br>
    <br>
    <blockquote type=3D"cite">
      <div class=3D"gmail_quote">
        <div><br>
        </div>
        <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border=
-left:1px #ccc solid;padding-left:1ex">
          <div bgcolor=3D"#FFFFFF" text=3D"#000000">
            <div><br>
              <br>
              <blockquote type=3D"cite">
                <div class=3D"gmail_quote">
                  <div><br>
                  </div>
                  <div>i attach 0f:10.0 to a domU ubuntu machine,=A0xm
                    pci-attach ubuntu 0f:10.0</div>
                  <div><br>
                  </div>
                  <div>on my dom0 machine, i can ping other machine
                    using eth2, (implying PF on eth2 is active)</div>
                  <div><br>
                  </div>
                  <div>on my domU machine, when i attach the virtual
                    function, i get the following messages</div>
                  <div><br>
                  </div>
                  <div>
                    <div><a href=3D"tel:%5B%202282.688356" value=3D"+122826=
88356" target=3D"_blank">[
                        2282.688356</a>] ixgbevf 0000:00:00.0: Xen PCI
                      mapped GSI0 to IRQ28</div>
                    <div><a href=3D"tel:%5B%202282.688470" value=3D"+122826=
88470" target=3D"_blank">[
                        2282.688470</a>] ixgbevf 0000:00:00.0: setting
                      latency timer to 64</div>
                    <div><a href=3D"tel:%5B%202282.690187" value=3D"+122826=
90187" target=3D"_blank">[
                        2282.690187</a>] ixgbevf 0000:00:00.0: PF still
                      in reset state, assigning new address</div>
                  </div>
                  <div><br>
                  </div>
                  <div>while PF on eth2 is =A0there and active, since i
                    can ping other machine.</div>
                  <div><br>
                  </div>
                  <div>Now, when I try to bring up the VF interface on
                    domU,=A0</div>
                  <div>=A0i get the following error</div>
                  <div> <br>
                  </div>
                  <div>
                    <div>=A02476.295582] Unable to start - perhaps the PF
                      Driver isn&#39;t up yet</div>
                    <div>SIOCSIFFLAGS: Network is down</div>
                    <div>[ 2476.296917] Unable to start - perhaps the PF
                      Driver isn&#39;t up yet</div>
                    <div> SIOCSIFFLAGS: Network is down</div>
                  </div>
                  <div><br>
                  </div>
                  <div>and on dmesg on domU,=A0</div>
                  <div>
                    <div>[ 2476.295582] Unable to start - perhaps the PF
                      Driver isn&#39;t up yet</div>
                    <div>[ 2476.296917] Unable to start - perhaps the PF
                      Driver isn&#39;t up yet</div>
                  </div>
                  <div><br>
                  </div>
                  <div><br>
                  </div>
                  <div>Any thoughts on what could be wrong? I have been
                    struggling with this for quite some time</div>
                  <div>and would really appreciate your thoughts on it.</di=
v>
                  <div><br>
                  </div>
                  <div>Thanks,=A0</div>
                  <span><font color=3D"#888888">
                      <div>Ashok</div>
                      <div><br>
                      </div>
                      <div><br>
                      </div>
                      <div><br>
                      </div>
                      <div><br>
                      </div>
                      <div><br>
                      </div>
                    </font></span></div>
                <br>
              </blockquote>
              <br>
            </div>
            <span><font color=3D"#888888">
                <pre cols=3D"72">--=20
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: <a href=3D"tel:%2B44%20%280%291223%20225%20900" value=3D"+441223225900" =
target=3D"_blank">+44 (0)1223 225 900</a>, <a href=3D"http://www.citrix.com=
" target=3D"_blank">http://www.citrix.com</a></pre>
              </font></span></div>
          <br>
          _______________________________________________<br>
          Xen-devel mailing list<br>
          <a href=3D"mailto:Xen-devel@lists.xen.org" target=3D"_blank">Xen-=
devel@lists.xen.org</a><br>
          <a href=3D"http://lists.xen.org/xen-devel" target=3D"_blank">http=
://lists.xen.org/xen-devel</a><br>
          <br>
        </blockquote>
      </div>
      <br>
    </blockquote>
    <br>
    <pre cols=3D"72">--=20
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: <a href=3D"tel:%2B44%20%280%291223%20225%20900" value=3D"+441223225900" =
target=3D"_blank">+44 (0)1223 225 900</a>, <a href=3D"http://www.citrix.com=
" target=3D"_blank">http://www.citrix.com</a></pre>
  </div></div></div>

</blockquote></div><br>

--e89a8fb1ffc2a69dac04c956ac5c--


--===============3163611272199496851==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

--===============3163611272199496851==--

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Ashok Anand <ashok.anand@gmail.com>
Subject: Re: issue using SRIOV "Unable to start - perhaps the PF
 driver is not up yet", while PF driver is actually up
Date: Tue, 11 Sep 2012 00:02:47 +0530
Message-ID: <CALQsBOUAv_kPuAjbvyPZs90rGM7ZOBjEctC6iMuf6c+CL+VkDw@mail.gmail.com>
References: <CALQsBOWDE_KqpntG8WJo86omLP3+rhQ0LDMW5vB752ubpF1FAA@mail.gmail.com>
	<CALQsBOXFVM+cdhmke5JDJmns2XUpHjyDOVN6R5qrrY9nTOR3SQ@mail.gmail.com>
	<504DC059.4070003@citrix.com>
	<CALQsBOV0EsUY+eYWqtL-FVuTOWOk_MLPnC2CYYPhNU9t=sxt9g@mail.gmail.com>
	<504DC334.6060201@citrix.com>
	<CALQsBOXqT39-ZkOuTT-c58qUW6dihuXWfUi08YzjmgJzaW9YAA@mail.gmail.com>
Mime-Version: 1.0
Content-Type: multipart/mixed; boundary="===============8874624558075012550=="
Return-path: <xen-devel-bounces@lists.xen.org>
In-Reply-To: <CALQsBOXqT39-ZkOuTT-c58qUW6dihuXWfUi08YzjmgJzaW9YAA@mail.gmail.com>
List-Unsubscribe: <http://lists.xen.org/cgi-bin/mailman/options/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xen.org>
List-Help: <mailto:xen-devel-request@lists.xen.org?subject=help>
List-Subscribe: <http://lists.xen.org/cgi-bin/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=subscribe>
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: "xen-devel@lists.xen.org" <xen-devel@lists.xen.org>
List-Id: xen-devel@lists.xenproject.org

--===============8874624558075012550==
Content-Type: multipart/alternative; boundary=001517478a522e908004c95d2fd3

--001517478a522e908004c95d2fd3
Content-Type: text/plain; charset=ISO-8859-1

Any more thoughts on it? I will really appreciate it.

regards,
Ashok

On Mon, Sep 10, 2012 at 4:16 PM, Ashok Anand <ashok.anand@gmail.com> wrote:

>
> I am passing virtual functions. please see below ,based on the message
> from lspci.
>
>> What does lspci -vv for the physical function say?
>>
>> As for the physical function: it is unsafe to pass physical functions to
>> a non-trusted guest, as the physical function has complete control over the
>> all the virtual functions, even if they are passed through to other
>> guests.  For that reason, the physical function should remain in dom0 (or a
>> device driver domain if you are going for disaggregation).
>>
>
> here is the message lspci -vv | grep Eth gives me,
>
> 0f:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
> Network Connection (rev 01)
>  Subsystem: Intel Corporation Ethernet Server Adapter X520-2
> 0f:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
> Network Connection (rev 01)
>  Subsystem: Intel Corporation Ethernet Server Adapter X520-2
> 0f:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 0f:10.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 0f:10.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 0f:10.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 0f:11.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 0f:11.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 0f:11.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 0f:11.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
>
> Here, 0f:00.0 is the physical function, while 0f:10.0 and others are
> virtual functions -- so I am attaching virtual functions.
>
>
>
>
>>
>>
>>
>>
>>>
>>>
>>>  i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0
>>>
>>>  on my dom0 machine, i can ping other machine using eth2, (implying PF
>>> on eth2 is active)
>>>
>>>  on my domU machine, when i attach the virtual function, i get the
>>> following messages
>>>
>>>  [ 2282.688356 <%5B%202282.688356>] ixgbevf 0000:00:00.0: Xen PCI
>>> mapped GSI0 to IRQ28
>>> [ 2282.688470 <%5B%202282.688470>] ixgbevf 0000:00:00.0: setting
>>> latency timer to 64
>>> [ 2282.690187 <%5B%202282.690187>] ixgbevf 0000:00:00.0: PF still in
>>> reset state, assigning new address
>>>
>>>  while PF on eth2 is  there and active, since i can ping other machine.
>>>
>>>  Now, when I try to bring up the VF interface on domU,
>>>  i get the following error
>>>
>>>   2476.295582] Unable to start - perhaps the PF Driver isn't up yet
>>> SIOCSIFFLAGS: Network is down
>>> [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
>>>  SIOCSIFFLAGS: Network is down
>>>
>>>  and on dmesg on domU,
>>>  [ 2476.295582] Unable to start - perhaps the PF Driver isn't up yet
>>> [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
>>>
>>>
>>>  Any thoughts on what could be wrong? I have been struggling with this
>>> for quite some time
>>> and would really appreciate your thoughts on it.
>>>
>>>  Thanks,
>>>  Ashok
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>  --
>>> Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>>> T: +44 (0)1223 225 900, http://www.citrix.com
>>>
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xen.org
>>> http://lists.xen.org/xen-devel
>>>
>>>
>>
>> --
>> Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>> T: +44 (0)1223 225 900, http://www.citrix.com
>>
>>
>

--001517478a522e908004c95d2fd3
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Any more thoughts on it? I will really appreciate it.<div><br></div><div>re=
gards,</div><div>Ashok<br><br><div class=3D"gmail_quote">On Mon, Sep 10, 20=
12 at 4:16 PM, Ashok Anand <span dir=3D"ltr">&lt;<a href=3D"mailto:ashok.an=
and@gmail.com" target=3D"_blank">ashok.anand@gmail.com</a>&gt;</span> wrote=
:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div class=3D"gmail_quote"><div><br></div><d=
iv>I am passing virtual functions. please see below ,based on the message f=
rom lspci.=A0</div>
<div class=3D"im"><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .=
8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor=3D"#FFFFFF" text=3D"#000000">What does lspci -vv for the physi=
cal function say?<br>
    <br>
    As for the physical function: it is unsafe to pass physical
    functions to a non-trusted guest, as the physical function has
    complete control over the all the virtual functions, even if they
    are passed through to other guests.=A0 For that reason, the physical
    function should remain in dom0 (or a device driver domain if you are
    going for disaggregation).</div></blockquote><div><br></div></div><div>=
here is the message lspci -vv | grep Eth gives me,=A0</div><div><br></div><=
div><div>0f:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit =
SFI/SFP+ Network Connection (rev 01)</div>

<div><span style=3D"white-space:pre-wrap">	</span>Subsystem: Intel Corporat=
ion Ethernet Server Adapter X520-2</div><div>0f:00.1 Ethernet controller: I=
ntel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)</d=
iv>

<div><span style=3D"white-space:pre-wrap">	</span>Subsystem: Intel Corporat=
ion Ethernet Server Adapter X520-2</div><div>0f:10.0 Ethernet controller: I=
ntel Corporation 82599 Ethernet Controller Virtual Function (rev 01)</div>

<div>0f:10.2 Ethernet controller: Intel Corporation 82599 Ethernet Controll=
er Virtual Function (rev 01)</div><div>0f:10.4 Ethernet controller: Intel C=
orporation 82599 Ethernet Controller Virtual Function (rev 01)</div><div>

0f:10.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller Vi=
rtual Function (rev 01)</div><div>0f:11.0 Ethernet controller: Intel Corpor=
ation 82599 Ethernet Controller Virtual Function (rev 01)</div><div>0f:11.2=
 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual F=
unction (rev 01)</div>

<div>0f:11.4 Ethernet controller: Intel Corporation 82599 Ethernet Controll=
er Virtual Function (rev 01)</div><div>0f:11.6 Ethernet controller: Intel C=
orporation 82599 Ethernet Controller Virtual Function (rev 01)</div></div>

<div><br></div><div>Here, 0f:00.0 is the physical function, while 0f:10.0 a=
nd others are virtual functions -- so I am attaching virtual functions.=A0<=
/div><div><div class=3D"h5"><div><br></div><div><br></div><div>=A0</div><bl=
ockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #=
ccc solid;padding-left:1ex">

<div bgcolor=3D"#FFFFFF" text=3D"#000000"><div><div><br>
    <br>
    <blockquote type=3D"cite">
      <div class=3D"gmail_quote">
        <div><br>
        </div>
        <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border=
-left:1px #ccc solid;padding-left:1ex">
          <div bgcolor=3D"#FFFFFF" text=3D"#000000">
            <div><br>
              <br>
              <blockquote type=3D"cite">
                <div class=3D"gmail_quote">
                  <div><br>
                  </div>
                  <div>i attach 0f:10.0 to a domU ubuntu machine,=A0xm
                    pci-attach ubuntu 0f:10.0</div>
                  <div><br>
                  </div>
                  <div>on my dom0 machine, i can ping other machine
                    using eth2, (implying PF on eth2 is active)</div>
                  <div><br>
                  </div>
                  <div>on my domU machine, when i attach the virtual
                    function, i get the following messages</div>
                  <div><br>
                  </div>
                  <div>
                    <div><a href=3D"tel:%5B%202282.688356" value=3D"+122826=
88356" target=3D"_blank">[
                        2282.688356</a>] ixgbevf 0000:00:00.0: Xen PCI
                      mapped GSI0 to IRQ28</div>
                    <div><a href=3D"tel:%5B%202282.688470" value=3D"+122826=
88470" target=3D"_blank">[
                        2282.688470</a>] ixgbevf 0000:00:00.0: setting
                      latency timer to 64</div>
                    <div><a href=3D"tel:%5B%202282.690187" value=3D"+122826=
90187" target=3D"_blank">[
                        2282.690187</a>] ixgbevf 0000:00:00.0: PF still
                      in reset state, assigning new address</div>
                  </div>
                  <div><br>
                  </div>
                  <div>while PF on eth2 is =A0there and active, since i
                    can ping other machine.</div>
                  <div><br>
                  </div>
                  <div>Now, when I try to bring up the VF interface on
                    domU,=A0</div>
                  <div>=A0i get the following error</div>
                  <div> <br>
                  </div>
                  <div>
                    <div>=A02476.295582] Unable to start - perhaps the PF
                      Driver isn&#39;t up yet</div>
                    <div>SIOCSIFFLAGS: Network is down</div>
                    <div>[ 2476.296917] Unable to start - perhaps the PF
                      Driver isn&#39;t up yet</div>
                    <div> SIOCSIFFLAGS: Network is down</div>
                  </div>
                  <div><br>
                  </div>
                  <div>and on dmesg on domU,=A0</div>
                  <div>
                    <div>[ 2476.295582] Unable to start - perhaps the PF
                      Driver isn&#39;t up yet</div>
                    <div>[ 2476.296917] Unable to start - perhaps the PF
                      Driver isn&#39;t up yet</div>
                  </div>
                  <div><br>
                  </div>
                  <div><br>
                  </div>
                  <div>Any thoughts on what could be wrong? I have been
                    struggling with this for quite some time</div>
                  <div>and would really appreciate your thoughts on it.</di=
v>
                  <div><br>
                  </div>
                  <div>Thanks,=A0</div>
                  <span><font color=3D"#888888">
                      <div>Ashok</div>
                      <div><br>
                      </div>
                      <div><br>
                      </div>
                      <div><br>
                      </div>
                      <div><br>
                      </div>
                      <div><br>
                      </div>
                    </font></span></div>
                <br>
              </blockquote>
              <br>
            </div>
            <span><font color=3D"#888888">
                <pre cols=3D"72">--=20
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: <a href=3D"tel:%2B44%20%280%291223%20225%20900" value=3D"+441223225900" =
target=3D"_blank">+44 (0)1223 225 900</a>, <a href=3D"http://www.citrix.com=
" target=3D"_blank">http://www.citrix.com</a></pre>
              </font></span></div>
          <br>
          _______________________________________________<br>
          Xen-devel mailing list<br>
          <a href=3D"mailto:Xen-devel@lists.xen.org" target=3D"_blank">Xen-=
devel@lists.xen.org</a><br>
          <a href=3D"http://lists.xen.org/xen-devel" target=3D"_blank">http=
://lists.xen.org/xen-devel</a><br>
          <br>
        </blockquote>
      </div>
      <br>
    </blockquote>
    <br>
    <pre cols=3D"72">--=20
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: <a href=3D"tel:%2B44%20%280%291223%20225%20900" value=3D"+441223225900" =
target=3D"_blank">+44 (0)1223 225 900</a>, <a href=3D"http://www.citrix.com=
" target=3D"_blank">http://www.citrix.com</a></pre>
  </div></div></div>

</blockquote></div></div></div><br>
</blockquote></div><br></div>

--001517478a522e908004c95d2fd3--


--===============8874624558075012550==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

--===============8874624558075012550==--

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: issue using SRIOV "Unable to start - perhaps the PF
 driver is not up yet", while PF driver is actually up
Date: Tue, 11 Sep 2012 10:32:25 +0100
Message-ID: <504F0529.7010204@citrix.com>
References: <CALQsBOWDE_KqpntG8WJo86omLP3+rhQ0LDMW5vB752ubpF1FAA@mail.gmail.com>
	<CALQsBOXFVM+cdhmke5JDJmns2XUpHjyDOVN6R5qrrY9nTOR3SQ@mail.gmail.com>
	<504DC059.4070003@citrix.com>
	<CALQsBOV0EsUY+eYWqtL-FVuTOWOk_MLPnC2CYYPhNU9t=sxt9g@mail.gmail.com>
	<504DC334.6060201@citrix.com>
	<CALQsBOXqT39-ZkOuTT-c58qUW6dihuXWfUi08YzjmgJzaW9YAA@mail.gmail.com>
	<CALQsBOUAv_kPuAjbvyPZs90rGM7ZOBjEctC6iMuf6c+CL+VkDw@mail.gmail.com>
Mime-Version: 1.0
Content-Type: multipart/mixed; boundary="===============0887041430581390211=="
Return-path: <xen-devel-bounces@lists.xen.org>
In-Reply-To: <CALQsBOUAv_kPuAjbvyPZs90rGM7ZOBjEctC6iMuf6c+CL+VkDw@mail.gmail.com>
List-Unsubscribe: <http://lists.xen.org/cgi-bin/mailman/options/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xen.org>
List-Help: <mailto:xen-devel-request@lists.xen.org?subject=help>
List-Subscribe: <http://lists.xen.org/cgi-bin/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=subscribe>
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: xen-devel@lists.xen.org
List-Id: xen-devel@lists.xenproject.org

--===============0887041430581390211==
Content-Type: multipart/alternative;
	boundary="------------030307030407000205090902"

--------------030307030407000205090902
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 7bit


On 10/09/12 19:32, Ashok Anand wrote:
> Any more thoughts on it? I will really appreciate it.

Sorry - I am debugging a memory corruption issue, so not paying too
close attention to emails.

Have you ensured that the pif is up in dom0 before trying to use a vif?

~Andrew

>
> regards,
> Ashok
>
> On Mon, Sep 10, 2012 at 4:16 PM, Ashok Anand <ashok.anand@gmail.com
> <mailto:ashok.anand@gmail.com>> wrote:
>
>
>     I am passing virtual functions. please see below ,based on the
>     message from lspci. 
>
>         What does lspci -vv for the physical function say?
>
>         As for the physical function: it is unsafe to pass physical
>         functions to a non-trusted guest, as the physical function has
>         complete control over the all the virtual functions, even if
>         they are passed through to other guests.  For that reason, the
>         physical function should remain in dom0 (or a device driver
>         domain if you are going for disaggregation).
>
>
>     here is the message lspci -vv | grep Eth gives me, 
>
>     0f:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
>     SFI/SFP+ Network Connection (rev 01)
>     Subsystem: Intel Corporation Ethernet Server Adapter X520-2
>     0f:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
>     SFI/SFP+ Network Connection (rev 01)
>     Subsystem: Intel Corporation Ethernet Server Adapter X520-2
>     0f:10.0 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>     0f:10.2 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>     0f:10.4 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>     0f:10.6 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>     0f:11.0 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>     0f:11.2 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>     0f:11.4 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>     0f:11.6 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>
>     Here, 0f:00.0 is the physical function, while 0f:10.0 and others
>     are virtual functions -- so I am attaching virtual functions. 
>
>
>      
>
>
>
>>
>>
>>
>>>
>>>             i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach
>>>             ubuntu 0f:10.0
>>>
>>>             on my dom0 machine, i can ping other machine using eth2,
>>>             (implying PF on eth2 is active)
>>>
>>>             on my domU machine, when i attach the virtual function,
>>>             i get the following messages
>>>
>>>             [ 2282.688356 <tel:%5B%202282.688356>] ixgbevf
>>>             0000:00:00.0: Xen PCI mapped GSI0 to IRQ28
>>>             [ 2282.688470 <tel:%5B%202282.688470>] ixgbevf
>>>             0000:00:00.0: setting latency timer to 64
>>>             [ 2282.690187 <tel:%5B%202282.690187>] ixgbevf
>>>             0000:00:00.0: PF still in reset state, assigning new address
>>>
>>>             while PF on eth2 is  there and active, since i can ping
>>>             other machine.
>>>
>>>             Now, when I try to bring up the VF interface on domU, 
>>>              i get the following error
>>>
>>>              2476.295582] Unable to start - perhaps the PF Driver
>>>             isn't up yet
>>>             SIOCSIFFLAGS: Network is down
>>>             [ 2476.296917] Unable to start - perhaps the PF Driver
>>>             isn't up yet
>>>             SIOCSIFFLAGS: Network is down
>>>
>>>             and on dmesg on domU, 
>>>             [ 2476.295582] Unable to start - perhaps the PF Driver
>>>             isn't up yet
>>>             [ 2476.296917] Unable to start - perhaps the PF Driver
>>>             isn't up yet
>>>
>>>
>>>             Any thoughts on what could be wrong? I have been
>>>             struggling with this for quite some time
>>>             and would really appreciate your thoughts on it.
>>>
>>>             Thanks, 
>>>             Ashok
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>             -- 
>>             Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>>             T: +44 (0)1223 225 900 <tel:%2B44%20%280%291223%20225%20900>, http://www.citrix.com
>>
>>
>>             _______________________________________________
>>             Xen-devel mailing list
>>             Xen-devel@lists.xen.org <mailto:Xen-devel@lists.xen.org>
>>             http://lists.xen.org/xen-devel
>>
>>
>
>         -- 
>         Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>         T: +44 (0)1223 225 900 <tel:%2B44%20%280%291223%20225%20900>, http://www.citrix.com
>
>
>

-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com


--------------030307030407000205090902
Content-Type: text/html; charset="ISO-8859-1"
Content-Transfer-Encoding: 7bit

<html>
  <head>
    <meta content="text/html; charset=ISO-8859-1"
      http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <br>
    <div class="moz-cite-prefix">On 10/09/12 19:32, Ashok Anand wrote:<br>
    </div>
    <blockquote
cite="mid:CALQsBOUAv_kPuAjbvyPZs90rGM7ZOBjEctC6iMuf6c+CL+VkDw@mail.gmail.com"
      type="cite">Any more thoughts on it? I will really appreciate it.</blockquote>
    <br>
    Sorry - I am debugging a memory corruption issue, so not paying too
    close attention to emails.<br>
    <br>
    Have you ensured that the pif is up in dom0 before trying to use a
    vif?<br>
    <br>
    ~Andrew<br>
    <br>
    <blockquote
cite="mid:CALQsBOUAv_kPuAjbvyPZs90rGM7ZOBjEctC6iMuf6c+CL+VkDw@mail.gmail.com"
      type="cite">
      <div><br>
      </div>
      <div>regards,</div>
      <div>Ashok<br>
        <br>
        <div class="gmail_quote">On Mon, Sep 10, 2012 at 4:16 PM, Ashok
          Anand <span dir="ltr">&lt;<a moz-do-not-send="true"
              href="mailto:ashok.anand@gmail.com" target="_blank">ashok.anand@gmail.com</a>&gt;</span>
          wrote:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0
            .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div class="gmail_quote">
              <div><br>
              </div>
              <div>I am passing virtual functions. please see below
                ,based on the message from lspci.&nbsp;</div>
              <div class="im">
                <blockquote class="gmail_quote" style="margin:0 0 0
                  .8ex;border-left:1px #ccc solid;padding-left:1ex">
                  <div bgcolor="#FFFFFF" text="#000000">What does lspci
                    -vv for the physical function say?<br>
                    <br>
                    As for the physical function: it is unsafe to pass
                    physical functions to a non-trusted guest, as the
                    physical function has complete control over the all
                    the virtual functions, even if they are passed
                    through to other guests.&nbsp; For that reason, the
                    physical function should remain in dom0 (or a device
                    driver domain if you are going for disaggregation).</div>
                </blockquote>
                <div><br>
                </div>
              </div>
              <div>here is the message lspci -vv | grep Eth gives me,&nbsp;</div>
              <div><br>
              </div>
              <div>
                <div>0f:00.0 Ethernet controller: Intel Corporation
                  82599EB 10-Gigabit SFI/SFP+ Network Connection (rev
                  01)</div>
                <div><span style="white-space:pre-wrap"> </span>Subsystem:
                  Intel Corporation Ethernet Server Adapter X520-2</div>
                <div>0f:00.1 Ethernet controller: Intel Corporation
                  82599EB 10-Gigabit SFI/SFP+ Network Connection (rev
                  01)</div>
                <div><span style="white-space:pre-wrap"> </span>Subsystem:
                  Intel Corporation Ethernet Server Adapter X520-2</div>
                <div>0f:10.0 Ethernet controller: Intel Corporation
                  82599 Ethernet Controller Virtual Function (rev 01)</div>
                <div>0f:10.2 Ethernet controller: Intel Corporation
                  82599 Ethernet Controller Virtual Function (rev 01)</div>
                <div>0f:10.4 Ethernet controller: Intel Corporation
                  82599 Ethernet Controller Virtual Function (rev 01)</div>
                <div>
                  0f:10.6 Ethernet controller: Intel Corporation 82599
                  Ethernet Controller Virtual Function (rev 01)</div>
                <div>0f:11.0 Ethernet controller: Intel Corporation
                  82599 Ethernet Controller Virtual Function (rev 01)</div>
                <div>0f:11.2 Ethernet controller: Intel Corporation
                  82599 Ethernet Controller Virtual Function (rev 01)</div>
                <div>0f:11.4 Ethernet controller: Intel Corporation
                  82599 Ethernet Controller Virtual Function (rev 01)</div>
                <div>0f:11.6 Ethernet controller: Intel Corporation
                  82599 Ethernet Controller Virtual Function (rev 01)</div>
              </div>
              <div><br>
              </div>
              <div>Here, 0f:00.0 is the physical function, while 0f:10.0
                and others are virtual functions -- so I am attaching
                virtual functions.&nbsp;</div>
              <div>
                <div class="h5">
                  <div><br>
                  </div>
                  <div><br>
                  </div>
                  <div>&nbsp;</div>
                  <blockquote class="gmail_quote" style="margin:0 0 0
                    .8ex;border-left:1px #ccc solid;padding-left:1ex">
                    <div bgcolor="#FFFFFF" text="#000000">
                      <div>
                        <div><br>
                          <br>
                          <blockquote type="cite">
                            <div class="gmail_quote">
                              <div><br>
                              </div>
                              <blockquote class="gmail_quote"
                                style="margin:0 0 0 .8ex;border-left:1px
                                #ccc solid;padding-left:1ex">
                                <div bgcolor="#FFFFFF" text="#000000">
                                  <div><br>
                                    <br>
                                    <blockquote type="cite">
                                      <div class="gmail_quote">
                                        <div><br>
                                        </div>
                                        <div>i attach 0f:10.0 to a domU
                                          ubuntu machine,&nbsp;xm pci-attach
                                          ubuntu 0f:10.0</div>
                                        <div><br>
                                        </div>
                                        <div>on my dom0 machine, i can
                                          ping other machine using eth2,
                                          (implying PF on eth2 is
                                          active)</div>
                                        <div><br>
                                        </div>
                                        <div>on my domU machine, when i
                                          attach the virtual function, i
                                          get the following messages</div>
                                        <div><br>
                                        </div>
                                        <div>
                                          <div><a moz-do-not-send="true"
href="tel:%5B%202282.688356" value="+12282688356" target="_blank">[
                                              2282.688356</a>] ixgbevf
                                            0000:00:00.0: Xen PCI mapped
                                            GSI0 to IRQ28</div>
                                          <div><a moz-do-not-send="true"
href="tel:%5B%202282.688470" value="+12282688470" target="_blank">[
                                              2282.688470</a>] ixgbevf
                                            0000:00:00.0: setting
                                            latency timer to 64</div>
                                          <div><a moz-do-not-send="true"
href="tel:%5B%202282.690187" value="+12282690187" target="_blank">[
                                              2282.690187</a>] ixgbevf
                                            0000:00:00.0: PF still in
                                            reset state, assigning new
                                            address</div>
                                        </div>
                                        <div><br>
                                        </div>
                                        <div>while PF on eth2 is &nbsp;there
                                          and active, since i can ping
                                          other machine.</div>
                                        <div><br>
                                        </div>
                                        <div>Now, when I try to bring up
                                          the VF interface on domU,&nbsp;</div>
                                        <div>&nbsp;i get the following error</div>
                                        <div> <br>
                                        </div>
                                        <div>
                                          <div>&nbsp;2476.295582] Unable to
                                            start - perhaps the PF
                                            Driver isn't up yet</div>
                                          <div>SIOCSIFFLAGS: Network is
                                            down</div>
                                          <div>[ 2476.296917] Unable to
                                            start - perhaps the PF
                                            Driver isn't up yet</div>
                                          <div> SIOCSIFFLAGS: Network is
                                            down</div>
                                        </div>
                                        <div><br>
                                        </div>
                                        <div>and on dmesg on domU,&nbsp;</div>
                                        <div>
                                          <div>[ 2476.295582] Unable to
                                            start - perhaps the PF
                                            Driver isn't up yet</div>
                                          <div>[ 2476.296917] Unable to
                                            start - perhaps the PF
                                            Driver isn't up yet</div>
                                        </div>
                                        <div><br>
                                        </div>
                                        <div><br>
                                        </div>
                                        <div>Any thoughts on what could
                                          be wrong? I have been
                                          struggling with this for quite
                                          some time</div>
                                        <div>and would really appreciate
                                          your thoughts on it.</div>
                                        <div><br>
                                        </div>
                                        <div>Thanks,&nbsp;</div>
                                        <span><font color="#888888">
                                            <div>Ashok</div>
                                            <div><br>
                                            </div>
                                            <div><br>
                                            </div>
                                            <div><br>
                                            </div>
                                            <div><br>
                                            </div>
                                            <div><br>
                                            </div>
                                          </font></span></div>
                                      <br>
                                    </blockquote>
                                    <br>
                                  </div>
                                  <span><font color="#888888">
                                      <pre cols="72">-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: <a moz-do-not-send="true" href="tel:%2B44%20%280%291223%20225%20900" value="+441223225900" target="_blank">+44 (0)1223 225 900</a>, <a moz-do-not-send="true" href="http://www.citrix.com" target="_blank">http://www.citrix.com</a></pre>
                                    </font></span></div>
                                <br>
_______________________________________________<br>
                                Xen-devel mailing list<br>
                                <a moz-do-not-send="true"
                                  href="mailto:Xen-devel@lists.xen.org"
                                  target="_blank">Xen-devel@lists.xen.org</a><br>
                                <a moz-do-not-send="true"
                                  href="http://lists.xen.org/xen-devel"
                                  target="_blank">http://lists.xen.org/xen-devel</a><br>
                                <br>
                              </blockquote>
                            </div>
                            <br>
                          </blockquote>
                          <br>
                          <pre cols="72">-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: <a moz-do-not-send="true" href="tel:%2B44%20%280%291223%20225%20900" value="+441223225900" target="_blank">+44 (0)1223 225 900</a>, <a moz-do-not-send="true" href="http://www.citrix.com" target="_blank">http://www.citrix.com</a></pre>
                        </div>
                      </div>
                    </div>
                  </blockquote>
                </div>
              </div>
            </div>
            <br>
          </blockquote>
        </div>
        <br>
      </div>
    </blockquote>
    <br>
    <pre class="moz-signature" cols="72">-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, <a class="moz-txt-link-freetext" href="http://www.citrix.com">http://www.citrix.com</a></pre>
  </body>
</html>

--------------030307030407000205090902--


--===============0887041430581390211==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

--===============0887041430581390211==--

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Pasi =?iso-8859-1?Q?K=E4rkk=E4inen?= <pasik@iki.fi>
Subject: Re: issue using SRIOV "Unable to start - perhaps the PF
 driver is not up yet", while PF driver is actually up
Date: Tue, 11 Sep 2012 13:05:10 +0300
Message-ID: <20120911100510.GP8912@reaktio.net>
References: <CALQsBOWDE_KqpntG8WJo86omLP3+rhQ0LDMW5vB752ubpF1FAA@mail.gmail.com>
	<CALQsBOXFVM+cdhmke5JDJmns2XUpHjyDOVN6R5qrrY9nTOR3SQ@mail.gmail.com>
	<504DC059.4070003@citrix.com>
	<CALQsBOV0EsUY+eYWqtL-FVuTOWOk_MLPnC2CYYPhNU9t=sxt9g@mail.gmail.com>
	<504DC334.6060201@citrix.com>
	<CALQsBOXqT39-ZkOuTT-c58qUW6dihuXWfUi08YzjmgJzaW9YAA@mail.gmail.com>
	<CALQsBOUAv_kPuAjbvyPZs90rGM7ZOBjEctC6iMuf6c+CL+VkDw@mail.gmail.com>
	<504F0529.7010204@citrix.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: <xen-devel-bounces@lists.xen.org>
Content-Disposition: inline
In-Reply-To: <504F0529.7010204@citrix.com>
List-Unsubscribe: <http://lists.xen.org/cgi-bin/mailman/options/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xen.org>
List-Help: <mailto:xen-devel-request@lists.xen.org?subject=help>
List-Subscribe: <http://lists.xen.org/cgi-bin/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=subscribe>
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: xen-devel@lists.xen.org
List-Id: xen-devel@lists.xenproject.org

On Tue, Sep 11, 2012 at 10:32:25AM +0100, Andrew Cooper wrote:
>    On 10/09/12 19:32, Ashok Anand wrote:
> 
>      Any more thoughts on it? I will really appreciate it.
> 
>    Sorry - I am debugging a memory corruption issue, so not paying too close
>    attention to emails.
> 
>    Have you ensured that the pif is up in dom0 before trying to use a vif?
> 

Yes, "ifconfig ethX up" for the PF is required in dom0 before the VF works in the VM.

-- Pasi

>    ~Andrew
> 
>      regards,
>      Ashok
> 
>      On Mon, Sep 10, 2012 at 4:16 PM, Ashok Anand <[1]ashok.anand@gmail.com>
>      wrote:
> 
>        I am passing virtual functions. please see below ,based on the message
>        from lspci.
> 
>          What does lspci -vv for the physical function say?
> 
>          As for the physical function: it is unsafe to pass physical
>          functions to a non-trusted guest, as the physical function has
>          complete control over the all the virtual functions, even if they
>          are passed through to other guests.  For that reason, the physical
>          function should remain in dom0 (or a device driver domain if you are
>          going for disaggregation).
> 
>        here is the message lspci -vv | grep Eth gives me,
>        0f:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
>        SFI/SFP+ Network Connection (rev 01)
>        Subsystem: Intel Corporation Ethernet Server Adapter X520-2
>        0f:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
>        SFI/SFP+ Network Connection (rev 01)
>        Subsystem: Intel Corporation Ethernet Server Adapter X520-2
>        0f:10.0 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        0f:10.2 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        0f:10.4 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        0f:10.6 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        0f:11.0 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        0f:11.2 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        0f:11.4 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        0f:11.6 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        Here, 0f:00.0 is the physical function, while 0f:10.0 and others are
>        virtual functions -- so I am attaching virtual functions.
> 
> 
>                i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach
>                ubuntu 0f:10.0
>                on my dom0 machine, i can ping other machine using eth2,
>                (implying PF on eth2 is active)
>                on my domU machine, when i attach the virtual function, i get
>                the following messages
>                [2][ 2282.688356] ixgbevf 0000:00:00.0: Xen PCI mapped GSI0 to
>                IRQ28
>                [3][ 2282.688470] ixgbevf 0000:00:00.0: setting latency timer
>                to 64
>                [4][ 2282.690187] ixgbevf 0000:00:00.0: PF still in reset
>                state, assigning new address
>                while PF on eth2 is  there and active, since i can ping other
>                machine.
>                Now, when I try to bring up the VF interface on domU,
>                 i get the following error
>                 2476.295582] Unable to start - perhaps the PF Driver isn't up
>                yet
>                SIOCSIFFLAGS: Network is down
>                [ 2476.296917] Unable to start - perhaps the PF Driver isn't
>                up yet
>                SIOCSIFFLAGS: Network is down
>                and on dmesg on domU,
>                [ 2476.295582] Unable to start - perhaps the PF Driver isn't
>                up yet
>                [ 2476.296917] Unable to start - perhaps the PF Driver isn't
>                up yet
>                Any thoughts on what could be wrong? I have been struggling
>                with this for quite some time
>                and would really appreciate your thoughts on it.
>                Thanks,
>                Ashok
> 
>  --
>  Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>  T: [5]+44 (0)1223 225 900, [6]http://www.citrix.com
> 
>              _______________________________________________
>              Xen-devel mailing list
>              [7]Xen-devel@lists.xen.org
>              [8]http://lists.xen.org/xen-devel
> 
>  --
>  Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>  T: [9]+44 (0)1223 225 900, [10]http://www.citrix.com
> 
>  --
>  Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>  T: +44 (0)1223 225 900, [11]http://www.citrix.com
> 
> References
> 
>    Visible links
>    1. mailto:ashok.anand@gmail.com
>    2. file:///tmp/tel:%5B%202282.688356
>    3. file:///tmp/tel:%5B%202282.688470
>    4. file:///tmp/tel:%5B%202282.690187
>    5. file:///tmp/tel:%2B44%20%280%291223%20225%20900
>    6. http://www.citrix.com/
>    7. mailto:Xen-devel@lists.xen.org
>    8. http://lists.xen.org/xen-devel
>    9. file:///tmp/tel:%2B44%20%280%291223%20225%20900
>   10. http://www.citrix.com/
>   11. http://www.citrix.com/

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel