* getting VF link info seems to be broken in 3.9-rc8
@ 2013-04-24 13:08 Or Gerlitz
2013-04-24 15:05 ` Williams, Mitch A
2013-04-24 15:38 ` Greg Rose
0 siblings, 2 replies; 20+ messages in thread
From: Or Gerlitz @ 2013-04-24 13:08 UTC (permalink / raw)
To: David Miller, Greg Rose, Alexander Duyck; +Cc: netdev, Rony Efraim
[re-posting as of non text only content in the 1st post - sorry for the
spam] Rony Efraim <ronye@mellanox.com>observed a regression in 3.9-rc8
with respect to getting VFs link info from user space. Using latest
iproute2 we can easily get the VF info on 3.8.8 but not on the net git
which is now aligned on 3.9-rc8, will try to bisect it later
today/tomorrow, but for the sake of maybe someone has an insight of the
bug, reporting it now...
You can see below that for eth10 and eth11 which are ixgbe PFs, the VF
info is reported on 3.8.8 but not on 3.9-rc8, we see the same problem
with patches to mlx4 which implement ndo_get_vf_info which gives some
evidence that the problem isn't in the ixgbe driver, but rather
somewhere else, maybe the networking core, from quick looking on
net/core/rtnetlink.c I don't see there a smoking gun, anyway
net.git]# git log --oneline v3.8..v3.9-rc8 net/core/rtnetlink.c
88c5b5c rtnetlink: Call nlmsg_parse() with correct header length
fcca143 rtnetlink: fix error return code in rtnl_link_fill()
a5b8db9 rtnetlink: Mask the rta_type when range checking
84d73cd rtnl: fix info leak on RTM_GETLINK request for VF devices
b67bfe0 hlist: drop the node parameter from iterators
1690be6 bridge: Add vlan support to static neighbors
6cbdcee bridge: Dump vlan information from a bridge port
407af32 bridge: Add netlink interface to configure vlans on bridge ports
c5c3510 netns: fdb: allow unprivileged users to add/del fdb entries
2afb9b5 ethtool: set addr_assign_type to NET_ADDR_SET when addr is
passed on create
471cb5a bonding: remove usage of dev->master
898e506 rtnetlink: remove usage of dev->master
e7c3273 rtnl: use dev_set_mac_address() instead of plain ndo_
9a57247 rtnl: expose carrier value with possibility to set it
Or.
--> using 3.8.8
# lspci | grep Eth
01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
05:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
05:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
05:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
05:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
# cd /sys/class/net/
# ls -l
lrwxrwxrwx 1 root root 0 Apr 24 12:17 br0 -> ../../devices/virtual/net/br0
lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth0 ->
../../devices/pci0000:00/0000:00:01.0/0000:01:00.0/net/eth0
lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth1 ->
../../devices/pci0000:00/0000:00:01.0/0000:01:00.1/net/eth1
lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth10 ->
../../devices/pci0000:00/0000:00:09.0/0000:05:00.1/net/eth10
lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth11 ->
../../devices/pci0000:00/0000:00:09.0/0000:05:00.0/net/eth11
lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth16 ->
../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth16
lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth17 ->
../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth17
lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth18 ->
../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth18
lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth19 ->
../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth19
lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth20 ->
../../devices/pci0000:00/0000:00:09.0/0000:05:10.1/net/eth20
lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth22 ->
../../devices/pci0000:00/0000:00:09.0/0000:05:10.0/net/eth22
lrwxrwxrwx 1 root root 0 Apr 24 12:17 lo -> ../../devices/virtual/net/lo
# modprobe -r ixgbevf
# ip link set dev eth10 vf 0 mac 00:10:00:10:00:10
# ip link set dev eth11 vf 0 mac 00:11:00:11:00:11
# modprobe ixgbevf
# ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode
DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0
state UP mode DEFAULT qlen 1000
link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 00:25:90:36:9c:ed brd ff:ff:ff:ff:ff:ff
4: eth16: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 00:02:c9:e6:01:11 brd ff:ff:ff:ff:ff:ff
5: eth17: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 00:02:c9:e6:01:12 brd ff:ff:ff:ff:ff:ff
6: eth18: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 00:02:c9:e6:02:11 brd ff:ff:ff:ff:ff:ff
7: eth19: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 00:02:c9:e6:02:12 brd ff:ff:ff:ff:ff:ff
8: eth11: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state
DOWN mode DEFAULT qlen 1000
link/ether 00:1b:21:55:1d:00 brd ff:ff:ff:ff:ff:ff
vf 0 MAC 00:11:00:11:00:11, spoof checking on
9: eth10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
mode DORMANT qlen 1000
link/ether 00:1b:21:55:1d:01 brd ff:ff:ff:ff:ff:ff
vf 0 MAC 00:10:00:10:00:10, spoof checking on
12: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
UP mode DEFAULT
link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
23: eth22: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 00:11:00:11:00:11 brd ff:ff:ff:ff:ff:ff
24: eth20: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 00:10:00:10:00:10 brd ff:ff:ff:ff:ff:ff
--> using 3.9-rc8
# lspci | grep Eth
01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
05:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
05:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
05:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
05:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
# cd /sys/class/net/
# ls -l
lrwxrwxrwx 1 root root 0 Apr 24 11:48 br0 -> ../../devices/virtual/net/br0
lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth0 ->
../../devices/pci0000:00/0000:00:01.0/0000:01:00.0/net/eth0
lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth1 ->
../../devices/pci0000:00/0000:00:01.0/0000:01:00.1/net/eth1
lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth10 ->
../../devices/pci0000:00/0000:00:09.0/0000:05:00.1/net/eth10
lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth11 ->
../../devices/pci0000:00/0000:00:09.0/0000:05:00.0/net/eth11
lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth16 ->
../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth16
lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth17 ->
../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth17
lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth18 ->
../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth18
lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth19 ->
../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth19
lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth20 ->
../../devices/pci0000:00/0000:00:09.0/0000:05:10.1/net/eth20
lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth22 ->
../../devices/pci0000:00/0000:00:09.0/0000:05:10.0/net/eth22
lrwxrwxrwx 1 root root 0 Apr 24 11:47 lo -> ../../devices/virtual/net/lo
# ip link show <--- no sign for VFs
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode
DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0
state UP mode DEFAULT qlen 1000
link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 00:25:90:36:9c:ed brd ff:ff:ff:ff:ff:ff
4: eth11: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 00:1b:21:55:1d:00 brd ff:ff:ff:ff:ff:ff
5: eth10: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 00:1b:21:55:1d:01 brd ff:ff:ff:ff:ff:ff
6: eth16: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 00:02:c9:e6:01:11 brd ff:ff:ff:ff:ff:ff
7: eth17: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 00:02:c9:e6:01:12 brd ff:ff:ff:ff:ff:ff
8: eth18: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 00:02:c9:e6:02:11 brd ff:ff:ff:ff:ff:ff
9: eth19: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 00:02:c9:e6:02:12 brd ff:ff:ff:ff:ff:ff
10: eth22: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 3e:9c:76:28:d9:da brd ff:ff:ff:ff:ff:ff
11: eth20: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether ea:f6:42:a7:d9:40 brd ff:ff:ff:ff:ff:ff
12: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
UP mode DEFAULT
link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
^ permalink raw reply [flat|nested] 20+ messages in thread
* RE: getting VF link info seems to be broken in 3.9-rc8
2013-04-24 13:08 getting VF link info seems to be broken in 3.9-rc8 Or Gerlitz
@ 2013-04-24 15:05 ` Williams, Mitch A
2013-04-25 18:29 ` Alexander Duyck
2013-04-24 15:38 ` Greg Rose
1 sibling, 1 reply; 20+ messages in thread
From: Williams, Mitch A @ 2013-04-24 15:05 UTC (permalink / raw)
To: Or Gerlitz, David Miller, Rose, Gregory V, Duyck, Alexander H,
e1000-devel@lists.sourceforge.net
Cc: netdev, Rony Efraim
Adding the e1000-devel list.
> -----Original Message-----
> From: netdev-owner@vger.kernel.org [mailto:netdev-owner@vger.kernel.org] On
> Behalf Of Or Gerlitz
> Sent: Wednesday, April 24, 2013 6:08 AM
> To: David Miller; Rose, Gregory V; Duyck, Alexander H
> Cc: netdev; Rony Efraim
> Subject: getting VF link info seems to be broken in 3.9-rc8
>
> [re-posting as of non text only content in the 1st post - sorry for the
> spam] Rony Efraim <ronye@mellanox.com>observed a regression in 3.9-rc8
> with respect to getting VFs link info from user space. Using latest
> iproute2 we can easily get the VF info on 3.8.8 but not on the net git
> which is now aligned on 3.9-rc8, will try to bisect it later
> today/tomorrow, but for the sake of maybe someone has an insight of the
> bug, reporting it now...
>
> You can see below that for eth10 and eth11 which are ixgbe PFs, the VF
> info is reported on 3.8.8 but not on 3.9-rc8, we see the same problem
> with patches to mlx4 which implement ndo_get_vf_info which gives some
> evidence that the problem isn't in the ixgbe driver, but rather
> somewhere else, maybe the networking core, from quick looking on
> net/core/rtnetlink.c I don't see there a smoking gun, anyway
>
> net.git]# git log --oneline v3.8..v3.9-rc8 net/core/rtnetlink.c
> 88c5b5c rtnetlink: Call nlmsg_parse() with correct header length
> fcca143 rtnetlink: fix error return code in rtnl_link_fill()
> a5b8db9 rtnetlink: Mask the rta_type when range checking
> 84d73cd rtnl: fix info leak on RTM_GETLINK request for VF devices
> b67bfe0 hlist: drop the node parameter from iterators
> 1690be6 bridge: Add vlan support to static neighbors
> 6cbdcee bridge: Dump vlan information from a bridge port
> 407af32 bridge: Add netlink interface to configure vlans on bridge ports
> c5c3510 netns: fdb: allow unprivileged users to add/del fdb entries
> 2afb9b5 ethtool: set addr_assign_type to NET_ADDR_SET when addr is
> passed on create
> 471cb5a bonding: remove usage of dev->master
> 898e506 rtnetlink: remove usage of dev->master
> e7c3273 rtnl: use dev_set_mac_address() instead of plain ndo_
> 9a57247 rtnl: expose carrier value with possibility to set it
>
>
> Or.
>
> --> using 3.8.8
>
> # lspci | grep Eth
> 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
> Connection (rev 01)
> 01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network
> Connection (rev 01)
> 05:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
> SFI/SFP+ Network Connection (rev 01)
> 05:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
> SFI/SFP+ Network Connection (rev 01)
> 05:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 05:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
>
> # cd /sys/class/net/
> # ls -l
> lrwxrwxrwx 1 root root 0 Apr 24 12:17 br0 -> ../../devices/virtual/net/br0
> lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth0 ->
> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.0/net/eth0
> lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth1 ->
> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.1/net/eth1
> lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth10 ->
> ../../devices/pci0000:00/0000:00:09.0/0000:05:00.1/net/eth10
> lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth11 ->
> ../../devices/pci0000:00/0000:00:09.0/0000:05:00.0/net/eth11
> lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth16 ->
> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth16
> lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth17 ->
> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth17
> lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth18 ->
> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth18
> lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth19 ->
> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth19
> lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth20 ->
> ../../devices/pci0000:00/0000:00:09.0/0000:05:10.1/net/eth20
> lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth22 ->
> ../../devices/pci0000:00/0000:00:09.0/0000:05:10.0/net/eth22
> lrwxrwxrwx 1 root root 0 Apr 24 12:17 lo -> ../../devices/virtual/net/lo
>
>
>
> # modprobe -r ixgbevf
> # ip link set dev eth10 vf 0 mac 00:10:00:10:00:10
> # ip link set dev eth11 vf 0 mac 00:11:00:11:00:11
> # modprobe ixgbevf
> # ip link show
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode
> DEFAULT
> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0
> state UP mode DEFAULT qlen 1000
> link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
> 3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:25:90:36:9c:ed brd ff:ff:ff:ff:ff:ff
> 4: eth16: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:02:c9:e6:01:11 brd ff:ff:ff:ff:ff:ff
> 5: eth17: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:02:c9:e6:01:12 brd ff:ff:ff:ff:ff:ff
> 6: eth18: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:02:c9:e6:02:11 brd ff:ff:ff:ff:ff:ff
> 7: eth19: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:02:c9:e6:02:12 brd ff:ff:ff:ff:ff:ff
> 8: eth11: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state
> DOWN mode DEFAULT qlen 1000
> link/ether 00:1b:21:55:1d:00 brd ff:ff:ff:ff:ff:ff
> vf 0 MAC 00:11:00:11:00:11, spoof checking on
> 9: eth10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
> mode DORMANT qlen 1000
> link/ether 00:1b:21:55:1d:01 brd ff:ff:ff:ff:ff:ff
> vf 0 MAC 00:10:00:10:00:10, spoof checking on
> 12: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
> UP mode DEFAULT
> link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
> 23: eth22: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:11:00:11:00:11 brd ff:ff:ff:ff:ff:ff
> 24: eth20: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:10:00:10:00:10 brd ff:ff:ff:ff:ff:ff
>
>
> --> using 3.9-rc8
>
> # lspci | grep Eth
> 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
> Connection (rev 01)
> 01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network
> Connection (rev 01)
> 05:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
> SFI/SFP+ Network Connection (rev 01)
> 05:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
> SFI/SFP+ Network Connection (rev 01)
> 05:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 05:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
>
> # cd /sys/class/net/
> # ls -l
> lrwxrwxrwx 1 root root 0 Apr 24 11:48 br0 -> ../../devices/virtual/net/br0
> lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth0 ->
> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.0/net/eth0
> lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth1 ->
> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.1/net/eth1
> lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth10 ->
> ../../devices/pci0000:00/0000:00:09.0/0000:05:00.1/net/eth10
> lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth11 ->
> ../../devices/pci0000:00/0000:00:09.0/0000:05:00.0/net/eth11
> lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth16 ->
> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth16
> lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth17 ->
> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth17
> lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth18 ->
> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth18
> lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth19 ->
> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth19
> lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth20 ->
> ../../devices/pci0000:00/0000:00:09.0/0000:05:10.1/net/eth20
> lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth22 ->
> ../../devices/pci0000:00/0000:00:09.0/0000:05:10.0/net/eth22
> lrwxrwxrwx 1 root root 0 Apr 24 11:47 lo -> ../../devices/virtual/net/lo
>
> # ip link show <--- no sign for VFs
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode
> DEFAULT
> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0
> state UP mode DEFAULT qlen 1000
> link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
> 3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:25:90:36:9c:ed brd ff:ff:ff:ff:ff:ff
> 4: eth11: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:1b:21:55:1d:00 brd ff:ff:ff:ff:ff:ff
> 5: eth10: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:1b:21:55:1d:01 brd ff:ff:ff:ff:ff:ff
> 6: eth16: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:02:c9:e6:01:11 brd ff:ff:ff:ff:ff:ff
> 7: eth17: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:02:c9:e6:01:12 brd ff:ff:ff:ff:ff:ff
> 8: eth18: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:02:c9:e6:02:11 brd ff:ff:ff:ff:ff:ff
> 9: eth19: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:02:c9:e6:02:12 brd ff:ff:ff:ff:ff:ff
> 10: eth22: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 3e:9c:76:28:d9:da brd ff:ff:ff:ff:ff:ff
> 11: eth20: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether ea:f6:42:a7:d9:40 brd ff:ff:ff:ff:ff:ff
> 12: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
> UP mode DEFAULT
> link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: getting VF link info seems to be broken in 3.9-rc8
2013-04-24 13:08 getting VF link info seems to be broken in 3.9-rc8 Or Gerlitz
2013-04-24 15:05 ` Williams, Mitch A
@ 2013-04-24 15:38 ` Greg Rose
2013-04-24 16:01 ` Greg Rose
1 sibling, 1 reply; 20+ messages in thread
From: Greg Rose @ 2013-04-24 15:38 UTC (permalink / raw)
To: Or Gerlitz; +Cc: David Miller, Alexander Duyck, netdev, Rony Efraim
On Wed, 24 Apr 2013 16:08:22 +0300
Or Gerlitz <ogerlitz@mellanox.com> wrote:
> [re-posting as of non text only content in the 1st post - sorry for
> the spam] Rony Efraim <ronye@mellanox.com>observed a regression in
> 3.9-rc8 with respect to getting VFs link info from user space. Using
> latest iproute2 we can easily get the VF info on 3.8.8 but not on the
> net git which is now aligned on 3.9-rc8, will try to bisect it later
> today/tomorrow, but for the sake of maybe someone has an insight of
> the bug, reporting it now...
>
> You can see below that for eth10 and eth11 which are ixgbe PFs, the
> VF info is reported on 3.8.8 but not on 3.9-rc8, we see the same
> problem with patches to mlx4 which implement ndo_get_vf_info which
> gives some evidence that the problem isn't in the ixgbe driver, but
> rather somewhere else, maybe the networking core, from quick looking
> on net/core/rtnetlink.c I don't see there a smoking gun, anyway
I'll forward this to our validation engineers and see if they're seeing
the problem too.
- Greg
>
> net.git]# git log --oneline v3.8..v3.9-rc8 net/core/rtnetlink.c
> 88c5b5c rtnetlink: Call nlmsg_parse() with correct header length
> fcca143 rtnetlink: fix error return code in rtnl_link_fill()
> a5b8db9 rtnetlink: Mask the rta_type when range checking
> 84d73cd rtnl: fix info leak on RTM_GETLINK request for VF devices
> b67bfe0 hlist: drop the node parameter from iterators
> 1690be6 bridge: Add vlan support to static neighbors
> 6cbdcee bridge: Dump vlan information from a bridge port
> 407af32 bridge: Add netlink interface to configure vlans on bridge
> ports c5c3510 netns: fdb: allow unprivileged users to add/del fdb
> entries 2afb9b5 ethtool: set addr_assign_type to NET_ADDR_SET when
> addr is passed on create
> 471cb5a bonding: remove usage of dev->master
> 898e506 rtnetlink: remove usage of dev->master
> e7c3273 rtnl: use dev_set_mac_address() instead of plain ndo_
> 9a57247 rtnl: expose carrier value with possibility to set it
>
>
> Or.
>
> --> using 3.8.8
>
> # lspci | grep Eth
> 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
> Connection (rev 01)
> 01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network
> Connection (rev 01)
> 05:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
> SFI/SFP+ Network Connection (rev 01)
> 05:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
> SFI/SFP+ Network Connection (rev 01)
> 05:10.0 Ethernet controller: Intel Corporation 82599 Ethernet
> Controller Virtual Function (rev 01)
> 05:10.1 Ethernet controller: Intel Corporation 82599 Ethernet
> Controller Virtual Function (rev 01)
>
> # cd /sys/class/net/
> # ls -l
> lrwxrwxrwx 1 root root 0 Apr 24 12:17 br0
> -> ../../devices/virtual/net/br0 lrwxrwxrwx 1 root root 0 Apr 24
> 12:17 eth0
> -> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.0/net/eth0
> lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth1
> -> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.1/net/eth1
> lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth10
> -> ../../devices/pci0000:00/0000:00:09.0/0000:05:00.1/net/eth10
> lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth11
> -> ../../devices/pci0000:00/0000:00:09.0/0000:05:00.0/net/eth11
> lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth16
> -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth16
> lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth17
> -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth17
> lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth18
> -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth18
> lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth19
> -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth19
> lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth20
> -> ../../devices/pci0000:00/0000:00:09.0/0000:05:10.1/net/eth20
> lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth22
> -> ../../devices/pci0000:00/0000:00:09.0/0000:05:10.0/net/eth22
> lrwxrwxrwx 1 root root 0 Apr 24 12:17 lo
> -> ../../devices/virtual/net/lo
>
>
>
> # modprobe -r ixgbevf
> # ip link set dev eth10 vf 0 mac 00:10:00:10:00:10
> # ip link set dev eth11 vf 0 mac 00:11:00:11:00:11
> # modprobe ixgbevf
> # ip link show
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
> mode DEFAULT
> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master
> br0 state UP mode DEFAULT qlen 1000
> link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
> 3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:25:90:36:9c:ed brd ff:ff:ff:ff:ff:ff
> 4: eth16: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:02:c9:e6:01:11 brd ff:ff:ff:ff:ff:ff
> 5: eth17: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:02:c9:e6:01:12 brd ff:ff:ff:ff:ff:ff
> 6: eth18: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:02:c9:e6:02:11 brd ff:ff:ff:ff:ff:ff
> 7: eth19: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:02:c9:e6:02:12 brd ff:ff:ff:ff:ff:ff
> 8: eth11: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state
> DOWN mode DEFAULT qlen 1000
> link/ether 00:1b:21:55:1d:00 brd ff:ff:ff:ff:ff:ff
> vf 0 MAC 00:11:00:11:00:11, spoof checking on
> 9: eth10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state
> UP mode DORMANT qlen 1000
> link/ether 00:1b:21:55:1d:01 brd ff:ff:ff:ff:ff:ff
> vf 0 MAC 00:10:00:10:00:10, spoof checking on
> 12: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
> state UP mode DEFAULT
> link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
> 23: eth22: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:11:00:11:00:11 brd ff:ff:ff:ff:ff:ff
> 24: eth20: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:10:00:10:00:10 brd ff:ff:ff:ff:ff:ff
>
>
> --> using 3.9-rc8
>
> # lspci | grep Eth
> 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
> Connection (rev 01)
> 01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network
> Connection (rev 01)
> 05:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
> SFI/SFP+ Network Connection (rev 01)
> 05:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
> SFI/SFP+ Network Connection (rev 01)
> 05:10.0 Ethernet controller: Intel Corporation 82599 Ethernet
> Controller Virtual Function (rev 01)
> 05:10.1 Ethernet controller: Intel Corporation 82599 Ethernet
> Controller Virtual Function (rev 01)
>
> # cd /sys/class/net/
> # ls -l
> lrwxrwxrwx 1 root root 0 Apr 24 11:48 br0
> -> ../../devices/virtual/net/br0 lrwxrwxrwx 1 root root 0 Apr 24
> 11:48 eth0
> -> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.0/net/eth0
> lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth1
> -> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.1/net/eth1
> lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth10
> -> ../../devices/pci0000:00/0000:00:09.0/0000:05:00.1/net/eth10
> lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth11
> -> ../../devices/pci0000:00/0000:00:09.0/0000:05:00.0/net/eth11
> lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth16
> -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth16
> lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth17
> -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth17
> lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth18
> -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth18
> lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth19
> -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth19
> lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth20
> -> ../../devices/pci0000:00/0000:00:09.0/0000:05:10.1/net/eth20
> lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth22
> -> ../../devices/pci0000:00/0000:00:09.0/0000:05:10.0/net/eth22
> lrwxrwxrwx 1 root root 0 Apr 24 11:47 lo
> -> ../../devices/virtual/net/lo
>
> # ip link show <--- no sign for VFs
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
> mode DEFAULT
> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master
> br0 state UP mode DEFAULT qlen 1000
> link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
> 3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:25:90:36:9c:ed brd ff:ff:ff:ff:ff:ff
> 4: eth11: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:1b:21:55:1d:00 brd ff:ff:ff:ff:ff:ff
> 5: eth10: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:1b:21:55:1d:01 brd ff:ff:ff:ff:ff:ff
> 6: eth16: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:02:c9:e6:01:11 brd ff:ff:ff:ff:ff:ff
> 7: eth17: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:02:c9:e6:01:12 brd ff:ff:ff:ff:ff:ff
> 8: eth18: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:02:c9:e6:02:11 brd ff:ff:ff:ff:ff:ff
> 9: eth19: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:02:c9:e6:02:12 brd ff:ff:ff:ff:ff:ff
> 10: eth22: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 3e:9c:76:28:d9:da brd ff:ff:ff:ff:ff:ff
> 11: eth20: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether ea:f6:42:a7:d9:40 brd ff:ff:ff:ff:ff:ff
> 12: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
> state UP mode DEFAULT
> link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: getting VF link info seems to be broken in 3.9-rc8
2013-04-24 15:38 ` Greg Rose
@ 2013-04-24 16:01 ` Greg Rose
2013-04-24 16:07 ` Or Gerlitz
2013-04-24 16:08 ` Li, Sibai
0 siblings, 2 replies; 20+ messages in thread
From: Greg Rose @ 2013-04-24 16:01 UTC (permalink / raw)
To: Greg Rose
Cc: Or Gerlitz, David Miller, Alexander Duyck, netdev, Rony Efraim,
Sibai.li
On Wed, 24 Apr 2013 08:38:25 -0700
Greg Rose <gregory.v.rose@intel.com> wrote:
> On Wed, 24 Apr 2013 16:08:22 +0300
> Or Gerlitz <ogerlitz@mellanox.com> wrote:
>
> > [re-posting as of non text only content in the 1st post - sorry for
> > the spam] Rony Efraim <ronye@mellanox.com>observed a regression in
> > 3.9-rc8 with respect to getting VFs link info from user space. Using
> > latest iproute2 we can easily get the VF info on 3.8.8 but not on
> > the net git which is now aligned on 3.9-rc8, will try to bisect it
> > later today/tomorrow, but for the sake of maybe someone has an
> > insight of the bug, reporting it now...
> >
> > You can see below that for eth10 and eth11 which are ixgbe PFs, the
> > VF info is reported on 3.8.8 but not on 3.9-rc8, we see the same
> > problem with patches to mlx4 which implement ndo_get_vf_info which
> > gives some evidence that the problem isn't in the ixgbe driver, but
> > rather somewhere else, maybe the networking core, from quick looking
> > on net/core/rtnetlink.c I don't see there a smoking gun, anyway
>
> I'll forward this to our validation engineers and see if they're
> seeing the problem too.
>
> - Greg
From our validation engineer Sibai Li (added to response):
I tested 3.9.0-rc8, I have no problem to list vf info. But I didn’t run
in FPP mode. But I don't think it's a matter. From the info he gave
here, I didn’t see he loaded the vf driver when he ran in kernel
3.9.0-rc8, but he did load the driver in 3.8.8.
---------
I'd add that having the VF driver loaded shouldn't make any difference.
- Greg
>
> >
> > net.git]# git log --oneline v3.8..v3.9-rc8 net/core/rtnetlink.c
> > 88c5b5c rtnetlink: Call nlmsg_parse() with correct header length
> > fcca143 rtnetlink: fix error return code in rtnl_link_fill()
> > a5b8db9 rtnetlink: Mask the rta_type when range checking
> > 84d73cd rtnl: fix info leak on RTM_GETLINK request for VF devices
> > b67bfe0 hlist: drop the node parameter from iterators
> > 1690be6 bridge: Add vlan support to static neighbors
> > 6cbdcee bridge: Dump vlan information from a bridge port
> > 407af32 bridge: Add netlink interface to configure vlans on bridge
> > ports c5c3510 netns: fdb: allow unprivileged users to add/del fdb
> > entries 2afb9b5 ethtool: set addr_assign_type to NET_ADDR_SET when
> > addr is passed on create
> > 471cb5a bonding: remove usage of dev->master
> > 898e506 rtnetlink: remove usage of dev->master
> > e7c3273 rtnl: use dev_set_mac_address() instead of plain ndo_
> > 9a57247 rtnl: expose carrier value with possibility to set it
> >
> >
> > Or.
> >
> > --> using 3.8.8
> >
> > # lspci | grep Eth
> > 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit
> > Network Connection (rev 01)
> > 01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit
> > Network Connection (rev 01)
> > 05:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
> > SFI/SFP+ Network Connection (rev 01)
> > 05:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
> > SFI/SFP+ Network Connection (rev 01)
> > 05:10.0 Ethernet controller: Intel Corporation 82599 Ethernet
> > Controller Virtual Function (rev 01)
> > 05:10.1 Ethernet controller: Intel Corporation 82599 Ethernet
> > Controller Virtual Function (rev 01)
> >
> > # cd /sys/class/net/
> > # ls -l
> > lrwxrwxrwx 1 root root 0 Apr 24 12:17 br0
> > -> ../../devices/virtual/net/br0 lrwxrwxrwx 1 root root 0 Apr 24
> > 12:17 eth0
> > -> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.0/net/eth0
> > lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth1
> > -> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.1/net/eth1
> > lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth10
> > -> ../../devices/pci0000:00/0000:00:09.0/0000:05:00.1/net/eth10
> > lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth11
> > -> ../../devices/pci0000:00/0000:00:09.0/0000:05:00.0/net/eth11
> > lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth16
> > -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth16
> > lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth17
> > -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth17
> > lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth18
> > -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth18
> > lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth19
> > -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth19
> > lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth20
> > -> ../../devices/pci0000:00/0000:00:09.0/0000:05:10.1/net/eth20
> > lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth22
> > -> ../../devices/pci0000:00/0000:00:09.0/0000:05:10.0/net/eth22
> > lrwxrwxrwx 1 root root 0 Apr 24 12:17 lo
> > -> ../../devices/virtual/net/lo
> >
> >
> >
> > # modprobe -r ixgbevf
> > # ip link set dev eth10 vf 0 mac 00:10:00:10:00:10
> > # ip link set dev eth11 vf 0 mac 00:11:00:11:00:11
> > # modprobe ixgbevf
> > # ip link show
> > 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
> > mode DEFAULT
> > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> > 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master
> > br0 state UP mode DEFAULT qlen 1000
> > link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
> > 3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> > DEFAULT qlen 1000
> > link/ether 00:25:90:36:9c:ed brd ff:ff:ff:ff:ff:ff
> > 4: eth16: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> > DEFAULT qlen 1000
> > link/ether 00:02:c9:e6:01:11 brd ff:ff:ff:ff:ff:ff
> > 5: eth17: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> > DEFAULT qlen 1000
> > link/ether 00:02:c9:e6:01:12 brd ff:ff:ff:ff:ff:ff
> > 6: eth18: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> > DEFAULT qlen 1000
> > link/ether 00:02:c9:e6:02:11 brd ff:ff:ff:ff:ff:ff
> > 7: eth19: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> > DEFAULT qlen 1000
> > link/ether 00:02:c9:e6:02:12 brd ff:ff:ff:ff:ff:ff
> > 8: eth11: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq
> > state DOWN mode DEFAULT qlen 1000
> > link/ether 00:1b:21:55:1d:00 brd ff:ff:ff:ff:ff:ff
> > vf 0 MAC 00:11:00:11:00:11, spoof checking on
> > 9: eth10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state
> > UP mode DORMANT qlen 1000
> > link/ether 00:1b:21:55:1d:01 brd ff:ff:ff:ff:ff:ff
> > vf 0 MAC 00:10:00:10:00:10, spoof checking on
> > 12: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
> > state UP mode DEFAULT
> > link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
> > 23: eth22: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
> > mode DEFAULT qlen 1000
> > link/ether 00:11:00:11:00:11 brd ff:ff:ff:ff:ff:ff
> > 24: eth20: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
> > mode DEFAULT qlen 1000
> > link/ether 00:10:00:10:00:10 brd ff:ff:ff:ff:ff:ff
> >
> >
> > --> using 3.9-rc8
> >
> > # lspci | grep Eth
> > 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit
> > Network Connection (rev 01)
> > 01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit
> > Network Connection (rev 01)
> > 05:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
> > SFI/SFP+ Network Connection (rev 01)
> > 05:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
> > SFI/SFP+ Network Connection (rev 01)
> > 05:10.0 Ethernet controller: Intel Corporation 82599 Ethernet
> > Controller Virtual Function (rev 01)
> > 05:10.1 Ethernet controller: Intel Corporation 82599 Ethernet
> > Controller Virtual Function (rev 01)
> >
> > # cd /sys/class/net/
> > # ls -l
> > lrwxrwxrwx 1 root root 0 Apr 24 11:48 br0
> > -> ../../devices/virtual/net/br0 lrwxrwxrwx 1 root root 0 Apr 24
> > 11:48 eth0
> > -> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.0/net/eth0
> > lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth1
> > -> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.1/net/eth1
> > lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth10
> > -> ../../devices/pci0000:00/0000:00:09.0/0000:05:00.1/net/eth10
> > lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth11
> > -> ../../devices/pci0000:00/0000:00:09.0/0000:05:00.0/net/eth11
> > lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth16
> > -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth16
> > lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth17
> > -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth17
> > lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth18
> > -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth18
> > lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth19
> > -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth19
> > lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth20
> > -> ../../devices/pci0000:00/0000:00:09.0/0000:05:10.1/net/eth20
> > lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth22
> > -> ../../devices/pci0000:00/0000:00:09.0/0000:05:10.0/net/eth22
> > lrwxrwxrwx 1 root root 0 Apr 24 11:47 lo
> > -> ../../devices/virtual/net/lo
> >
> > # ip link show <--- no sign for VFs
> > 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
> > mode DEFAULT
> > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> > 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master
> > br0 state UP mode DEFAULT qlen 1000
> > link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
> > 3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> > DEFAULT qlen 1000
> > link/ether 00:25:90:36:9c:ed brd ff:ff:ff:ff:ff:ff
> > 4: eth11: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> > DEFAULT qlen 1000
> > link/ether 00:1b:21:55:1d:00 brd ff:ff:ff:ff:ff:ff
> > 5: eth10: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> > DEFAULT qlen 1000
> > link/ether 00:1b:21:55:1d:01 brd ff:ff:ff:ff:ff:ff
> > 6: eth16: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> > DEFAULT qlen 1000
> > link/ether 00:02:c9:e6:01:11 brd ff:ff:ff:ff:ff:ff
> > 7: eth17: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> > DEFAULT qlen 1000
> > link/ether 00:02:c9:e6:01:12 brd ff:ff:ff:ff:ff:ff
> > 8: eth18: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> > DEFAULT qlen 1000
> > link/ether 00:02:c9:e6:02:11 brd ff:ff:ff:ff:ff:ff
> > 9: eth19: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> > DEFAULT qlen 1000
> > link/ether 00:02:c9:e6:02:12 brd ff:ff:ff:ff:ff:ff
> > 10: eth22: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
> > mode DEFAULT qlen 1000
> > link/ether 3e:9c:76:28:d9:da brd ff:ff:ff:ff:ff:ff
> > 11: eth20: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
> > mode DEFAULT qlen 1000
> > link/ether ea:f6:42:a7:d9:40 brd ff:ff:ff:ff:ff:ff
> > 12: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
> > state UP mode DEFAULT
> > link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: getting VF link info seems to be broken in 3.9-rc8
2013-04-24 16:01 ` Greg Rose
@ 2013-04-24 16:07 ` Or Gerlitz
2013-04-24 16:12 ` Li, Sibai
2013-04-24 16:08 ` Li, Sibai
1 sibling, 1 reply; 20+ messages in thread
From: Or Gerlitz @ 2013-04-24 16:07 UTC (permalink / raw)
To: Greg Rose; +Cc: David Miller, Alexander Duyck, netdev, Rony Efraim, Sibai.li
On 24/04/2013 19:01, Greg Rose wrote:
> From our validation engineer Sibai Li (added to response):
>
> I tested 3.9.0-rc8, I have no problem to list vf info. But I didn’t run
> in FPP mode. But I don't think it's a matter. From the info he gave
> here, I didn’t see he loaded the vf driver when he ran in kernel
> 3.9.0-rc8, but he did load the driver in 3.8.8.
>
> ---------
>
> I'd add that having the VF driver loaded shouldn't make any difference.
I have tested again (this time the server is booted on net-next which
is 3.9-rc8+) and the VF driver IS loaded, but stillno sign for the VF in
the ip link show output
# lspci | grep Eth
01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
05:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
05:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
05:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
05:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
# lsmod | grep ixgbe
ixgbevf 34062 0
ixgbe 150783 0
hwmon 1839 1 ixgbe
mdio 4263 1 ixgbe
ptp 8400 2 ixgbe,igb
dca 6822 3 ixgbe,igb,ioatdma
# /mnt/upstream/tools/iproute2/ip/ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode
DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0
state UP mode DEFAULT qlen 1000
link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 00:25:90:36:9c:ed brd ff:ff:ff:ff:ff:ff
10: eth11: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 00:1b:21:55:1d:00 brd ff:ff:ff:ff:ff:ff
11: eth10: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 00:1b:21:55:1d:01 brd ff:ff:ff:ff:ff:ff
12: eth22: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 3a:20:bd:09:ce:9d brd ff:ff:ff:ff:ff:ff
13: eth20: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 52:42:81:a7:6e:da brd ff:ff:ff:ff:ff:ff
14: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
UP mode DEFAULT
link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
17: eth16: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state
DOWN mode DEFAULT qlen 1000
link/ether 00:02:c9:e6:01:11 brd ff:ff:ff:ff:ff:ff
18: eth17: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state
DOWN mode DEFAULT qlen 1000
link/ether 00:02:c9:e6:01:12 brd ff:ff:ff:ff:ff:ff
19: eth18: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 00:02:c9:e6:02:11 brd ff:ff:ff:ff:ff:ff
20: eth19: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 00:02:c9:e6:02:12 brd ff:ff:ff:ff:ff:ff
21: eth28: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 00:02:c9:e6:03:11 brd ff:ff:ff:ff:ff:ff
22: eth27: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT qlen 1000
link/ether 00:02:c9:e6:03:12 brd ff:ff:ff:ff:ff:ff
# pwd
/sys/class/net
# ls -l
total 0
lrwxrwxrwx 1 root root 0 Apr 24 18:44 br0 -> ../../devices/virtual/net/br0
lrwxrwxrwx 1 root root 0 Apr 24 18:44 eth0 ->
../../devices/pci0000:00/0000:00:01.0/0000:01:00.0/net/eth0
lrwxrwxrwx 1 root root 0 Apr 24 18:44 eth1 ->
../../devices/pci0000:00/0000:00:01.0/0000:01:00.1/net/eth1
lrwxrwxrwx 1 root root 0 Apr 24 18:45 eth10 ->
../../devices/pci0000:00/0000:00:09.0/0000:05:00.1/net/eth10
lrwxrwxrwx 1 root root 0 Apr 24 18:45 eth11 ->
../../devices/pci0000:00/0000:00:09.0/0000:05:00.0/net/eth11
lrwxrwxrwx 1 root root 0 Apr 24 18:51 eth16 ->
../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth16
lrwxrwxrwx 1 root root 0 Apr 24 18:51 eth17 ->
../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth17
lrwxrwxrwx 1 root root 0 Apr 24 18:51 eth18 ->
../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth18
lrwxrwxrwx 1 root root 0 Apr 24 18:51 eth19 ->
../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth19
lrwxrwxrwx 1 root root 0 Apr 24 18:44 eth20 ->
../../devices/pci0000:00/0000:00:09.0/0000:05:10.1/net/eth20
lrwxrwxrwx 1 root root 0 Apr 24 18:44 eth22 ->
../../devices/pci0000:00/0000:00:09.0/0000:05:10.0/net/eth22
lrwxrwxrwx 1 root root 0 Apr 24 18:51 eth27 ->
../../devices/pci0000:00/0000:00:07.0/0000:04:00.2/net/eth27
lrwxrwxrwx 1 root root 0 Apr 24 18:51 eth28 ->
../../devices/pci0000:00/0000:00:07.0/0000:04:00.2/net/eth28
lrwxrwxrwx 1 root root 0 Apr 24 18:44 lo -> ../../devices/virtual/net/lo
^ permalink raw reply [flat|nested] 20+ messages in thread
* RE: getting VF link info seems to be broken in 3.9-rc8
2013-04-24 16:01 ` Greg Rose
2013-04-24 16:07 ` Or Gerlitz
@ 2013-04-24 16:08 ` Li, Sibai
2013-04-24 16:34 ` Or Gerlitz
1 sibling, 1 reply; 20+ messages in thread
From: Li, Sibai @ 2013-04-24 16:08 UTC (permalink / raw)
To: Rose, Gregory V
Cc: Or Gerlitz, David Miller, Duyck, Alexander H, netdev, Rony Efraim
The vfs are listed, eth22/eth20. Just the mac address is different. Reload the vf driver to see if it corrects the thing.
Sibai
> -----Original Message-----
> From: Rose, Gregory V
> Sent: Wednesday, April 24, 2013 9:02 AM
> To: Rose, Gregory V
> Cc: Or Gerlitz; David Miller; Duyck, Alexander H; netdev; Rony Efraim; Li, Sibai
> Subject: Re: getting VF link info seems to be broken in 3.9-rc8
>
> On Wed, 24 Apr 2013 08:38:25 -0700
> Greg Rose <gregory.v.rose@intel.com> wrote:
>
> > On Wed, 24 Apr 2013 16:08:22 +0300
> > Or Gerlitz <ogerlitz@mellanox.com> wrote:
> >
> > > [re-posting as of non text only content in the 1st post - sorry for
> > > the spam] Rony Efraim <ronye@mellanox.com>observed a regression in
> > > 3.9-rc8 with respect to getting VFs link info from user space. Using
> > > latest iproute2 we can easily get the VF info on 3.8.8 but not on
> > > the net git which is now aligned on 3.9-rc8, will try to bisect it
> > > later today/tomorrow, but for the sake of maybe someone has an
> > > insight of the bug, reporting it now...
> > >
> > > You can see below that for eth10 and eth11 which are ixgbe PFs, the
> > > VF info is reported on 3.8.8 but not on 3.9-rc8, we see the same
> > > problem with patches to mlx4 which implement ndo_get_vf_info which
> > > gives some evidence that the problem isn't in the ixgbe driver, but
> > > rather somewhere else, maybe the networking core, from quick looking
> > > on net/core/rtnetlink.c I don't see there a smoking gun, anyway
> >
> > I'll forward this to our validation engineers and see if they're
> > seeing the problem too.
> >
> > - Greg
>
> From our validation engineer Sibai Li (added to response):
>
> I tested 3.9.0-rc8, I have no problem to list vf info. But I didn’t run in FPP mode.
> But I don't think it's a matter. From the info he gave here, I didn’t see he loaded
> the vf driver when he ran in kernel 3.9.0-rc8, but he did load the driver in 3.8.8.
>
> ---------
>
> I'd add that having the VF driver loaded shouldn't make any difference.
>
> - Greg
>
> >
> > >
> > > net.git]# git log --oneline v3.8..v3.9-rc8 net/core/rtnetlink.c
> > > 88c5b5c rtnetlink: Call nlmsg_parse() with correct header length
> > > fcca143 rtnetlink: fix error return code in rtnl_link_fill()
> > > a5b8db9 rtnetlink: Mask the rta_type when range checking 84d73cd
> > > rtnl: fix info leak on RTM_GETLINK request for VF devices
> > > b67bfe0 hlist: drop the node parameter from iterators
> > > 1690be6 bridge: Add vlan support to static neighbors 6cbdcee bridge:
> > > Dump vlan information from a bridge port
> > > 407af32 bridge: Add netlink interface to configure vlans on bridge
> > > ports c5c3510 netns: fdb: allow unprivileged users to add/del fdb
> > > entries 2afb9b5 ethtool: set addr_assign_type to NET_ADDR_SET when
> > > addr is passed on create 471cb5a bonding: remove usage of
> > > dev->master
> > > 898e506 rtnetlink: remove usage of dev->master
> > > e7c3273 rtnl: use dev_set_mac_address() instead of plain ndo_
> > > 9a57247 rtnl: expose carrier value with possibility to set it
> > >
> > >
> > > Or.
> > >
> > > --> using 3.8.8
> > >
> > > # lspci | grep Eth
> > > 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
> > > Connection (rev 01)
> > > 01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network
> > > Connection (rev 01)
> > > 05:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
> > > SFI/SFP+ Network Connection (rev 01)
> > > 05:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
> > > SFI/SFP+ Network Connection (rev 01)
> > > 05:10.0 Ethernet controller: Intel Corporation 82599 Ethernet
> > > Controller Virtual Function (rev 01)
> > > 05:10.1 Ethernet controller: Intel Corporation 82599 Ethernet
> > > Controller Virtual Function (rev 01)
> > >
> > > # cd /sys/class/net/
> > > # ls -l
> > > lrwxrwxrwx 1 root root 0 Apr 24 12:17 br0
> > > -> ../../devices/virtual/net/br0 lrwxrwxrwx 1 root root 0 Apr 24
> > > 12:17 eth0
> > > -> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.0/net/eth0
> > > lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth1
> > > -> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.1/net/eth1
> > > lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth10
> > > -> ../../devices/pci0000:00/0000:00:09.0/0000:05:00.1/net/eth10
> > > lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth11
> > > -> ../../devices/pci0000:00/0000:00:09.0/0000:05:00.0/net/eth11
> > > lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth16
> > > -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth16
> > > lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth17
> > > -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth17
> > > lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth18
> > > -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth18
> > > lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth19
> > > -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth19
> > > lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth20
> > > -> ../../devices/pci0000:00/0000:00:09.0/0000:05:10.1/net/eth20
> > > lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth22
> > > -> ../../devices/pci0000:00/0000:00:09.0/0000:05:10.0/net/eth22
> > > lrwxrwxrwx 1 root root 0 Apr 24 12:17 lo
> > > -> ../../devices/virtual/net/lo
> > >
> > >
> > >
> > > # modprobe -r ixgbevf
> > > # ip link set dev eth10 vf 0 mac 00:10:00:10:00:10 # ip link set dev
> > > eth11 vf 0 mac 00:11:00:11:00:11 # modprobe ixgbevf # ip link show
> > > 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state
> UNKNOWN
> > > mode DEFAULT
> > > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> > > 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq
> master
> > > br0 state UP mode DEFAULT qlen 1000
> > > link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
> > > 3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> > > DEFAULT qlen 1000
> > > link/ether 00:25:90:36:9c:ed brd ff:ff:ff:ff:ff:ff
> > > 4: eth16: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
> mode
> > > DEFAULT qlen 1000
> > > link/ether 00:02:c9:e6:01:11 brd ff:ff:ff:ff:ff:ff
> > > 5: eth17: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
> mode
> > > DEFAULT qlen 1000
> > > link/ether 00:02:c9:e6:01:12 brd ff:ff:ff:ff:ff:ff
> > > 6: eth18: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
> mode
> > > DEFAULT qlen 1000
> > > link/ether 00:02:c9:e6:02:11 brd ff:ff:ff:ff:ff:ff
> > > 7: eth19: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
> mode
> > > DEFAULT qlen 1000
> > > link/ether 00:02:c9:e6:02:12 brd ff:ff:ff:ff:ff:ff
> > > 8: eth11: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq
> > > state DOWN mode DEFAULT qlen 1000
> > > link/ether 00:1b:21:55:1d:00 brd ff:ff:ff:ff:ff:ff
> > > vf 0 MAC 00:11:00:11:00:11, spoof checking on
> > > 9: eth10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq
> state
> > > UP mode DORMANT qlen 1000
> > > link/ether 00:1b:21:55:1d:01 brd ff:ff:ff:ff:ff:ff
> > > vf 0 MAC 00:10:00:10:00:10, spoof checking on
> > > 12: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
> > > state UP mode DEFAULT
> > > link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
> > > 23: eth22: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
> mode
> > > DEFAULT qlen 1000
> > > link/ether 00:11:00:11:00:11 brd ff:ff:ff:ff:ff:ff
> > > 24: eth20: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
> mode
> > > DEFAULT qlen 1000
> > > link/ether 00:10:00:10:00:10 brd ff:ff:ff:ff:ff:ff
> > >
> > >
> > > --> using 3.9-rc8
> > >
> > > # lspci | grep Eth
> > > 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
> > > Connection (rev 01)
> > > 01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network
> > > Connection (rev 01)
> > > 05:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
> > > SFI/SFP+ Network Connection (rev 01)
> > > 05:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
> > > SFI/SFP+ Network Connection (rev 01)
> > > 05:10.0 Ethernet controller: Intel Corporation 82599 Ethernet
> > > Controller Virtual Function (rev 01)
> > > 05:10.1 Ethernet controller: Intel Corporation 82599 Ethernet
> > > Controller Virtual Function (rev 01)
> > >
> > > # cd /sys/class/net/
> > > # ls -l
> > > lrwxrwxrwx 1 root root 0 Apr 24 11:48 br0
> > > -> ../../devices/virtual/net/br0 lrwxrwxrwx 1 root root 0 Apr 24
> > > 11:48 eth0
> > > -> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.0/net/eth0
> > > lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth1
> > > -> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.1/net/eth1
> > > lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth10
> > > -> ../../devices/pci0000:00/0000:00:09.0/0000:05:00.1/net/eth10
> > > lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth11
> > > -> ../../devices/pci0000:00/0000:00:09.0/0000:05:00.0/net/eth11
> > > lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth16
> > > -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth16
> > > lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth17
> > > -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth17
> > > lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth18
> > > -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth18
> > > lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth19
> > > -> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth19
> > > lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth20
> > > -> ../../devices/pci0000:00/0000:00:09.0/0000:05:10.1/net/eth20
> > > lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth22
> > > -> ../../devices/pci0000:00/0000:00:09.0/0000:05:10.0/net/eth22
> > > lrwxrwxrwx 1 root root 0 Apr 24 11:47 lo
> > > -> ../../devices/virtual/net/lo
> > >
> > > # ip link show <--- no sign for VFs
> > > 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state
> UNKNOWN
> > > mode DEFAULT
> > > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> > > 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq
> master
> > > br0 state UP mode DEFAULT qlen 1000
> > > link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
> > > 3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> > > DEFAULT qlen 1000
> > > link/ether 00:25:90:36:9c:ed brd ff:ff:ff:ff:ff:ff
> > > 4: eth11: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
> mode
> > > DEFAULT qlen 1000
> > > link/ether 00:1b:21:55:1d:00 brd ff:ff:ff:ff:ff:ff
> > > 5: eth10: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
> mode
> > > DEFAULT qlen 1000
> > > link/ether 00:1b:21:55:1d:01 brd ff:ff:ff:ff:ff:ff
> > > 6: eth16: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
> mode
> > > DEFAULT qlen 1000
> > > link/ether 00:02:c9:e6:01:11 brd ff:ff:ff:ff:ff:ff
> > > 7: eth17: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
> mode
> > > DEFAULT qlen 1000
> > > link/ether 00:02:c9:e6:01:12 brd ff:ff:ff:ff:ff:ff
> > > 8: eth18: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
> mode
> > > DEFAULT qlen 1000
> > > link/ether 00:02:c9:e6:02:11 brd ff:ff:ff:ff:ff:ff
> > > 9: eth19: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
> mode
> > > DEFAULT qlen 1000
> > > link/ether 00:02:c9:e6:02:12 brd ff:ff:ff:ff:ff:ff
> > > 10: eth22: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
> mode
> > > DEFAULT qlen 1000
> > > link/ether 3e:9c:76:28:d9:da brd ff:ff:ff:ff:ff:ff
> > > 11: eth20: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
> mode
> > > DEFAULT qlen 1000
> > > link/ether ea:f6:42:a7:d9:40 brd ff:ff:ff:ff:ff:ff
> > > 12: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
> > > state UP mode DEFAULT
> > > link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe netdev" in
> > the body of a message to majordomo@vger.kernel.org More majordomo info
> > at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 20+ messages in thread
* RE: getting VF link info seems to be broken in 3.9-rc8
2013-04-24 16:07 ` Or Gerlitz
@ 2013-04-24 16:12 ` Li, Sibai
0 siblings, 0 replies; 20+ messages in thread
From: Li, Sibai @ 2013-04-24 16:12 UTC (permalink / raw)
To: Or Gerlitz, Rose, Gregory V
Cc: David Miller, Duyck, Alexander H, netdev, Rony Efraim
My guess is you load the PF driver after you loaded the VF. Please make sure you load the VF driver after the PF driver is loaded.
> -----Original Message-----
> From: Or Gerlitz [mailto:ogerlitz@mellanox.com]
> Sent: Wednesday, April 24, 2013 9:08 AM
> To: Rose, Gregory V
> Cc: David Miller; Duyck, Alexander H; netdev; Rony Efraim; Li, Sibai
> Subject: Re: getting VF link info seems to be broken in 3.9-rc8
>
> On 24/04/2013 19:01, Greg Rose wrote:
> > From our validation engineer Sibai Li (added to response):
> >
> > I tested 3.9.0-rc8, I have no problem to list vf info. But I didn’t
> > run in FPP mode. But I don't think it's a matter. From the info he
> > gave here, I didn’t see he loaded the vf driver when he ran in kernel
> > 3.9.0-rc8, but he did load the driver in 3.8.8.
> >
> > ---------
> >
> > I'd add that having the VF driver loaded shouldn't make any difference.
>
> I have tested again (this time the server is booted on net-next which is 3.9-rc8+)
> and the VF driver IS loaded, but stillno sign for the VF in the ip link show output
>
> # lspci | grep Eth
> 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
> Connection (rev 01)
> 01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network
> Connection (rev 01)
> 05:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
> Network Connection (rev 01)
> 05:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
> Network Connection (rev 01)
> 05:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual
> Function (rev 01)
> 05:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual
> Function (rev 01)
>
> # lsmod | grep ixgbe
> ixgbevf 34062 0
> ixgbe 150783 0
> hwmon 1839 1 ixgbe
> mdio 4263 1 ixgbe
> ptp 8400 2 ixgbe,igb
> dca 6822 3 ixgbe,igb,ioatdma
>
> # /mnt/upstream/tools/iproute2/ip/ip link show
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
> mode DEFAULT
> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master
> br0 state UP mode DEFAULT qlen 1000
> link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
> 3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:25:90:36:9c:ed brd ff:ff:ff:ff:ff:ff
> 10: eth11: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:1b:21:55:1d:00 brd ff:ff:ff:ff:ff:ff
> 11: eth10: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:1b:21:55:1d:01 brd ff:ff:ff:ff:ff:ff
> 12: eth22: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 3a:20:bd:09:ce:9d brd ff:ff:ff:ff:ff:ff
> 13: eth20: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 52:42:81:a7:6e:da brd ff:ff:ff:ff:ff:ff
> 14: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
> state UP mode DEFAULT
> link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
> 17: eth16: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state
> DOWN mode DEFAULT qlen 1000
> link/ether 00:02:c9:e6:01:11 brd ff:ff:ff:ff:ff:ff
> 18: eth17: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state
> DOWN mode DEFAULT qlen 1000
> link/ether 00:02:c9:e6:01:12 brd ff:ff:ff:ff:ff:ff
> 19: eth18: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:02:c9:e6:02:11 brd ff:ff:ff:ff:ff:ff
> 20: eth19: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:02:c9:e6:02:12 brd ff:ff:ff:ff:ff:ff
> 21: eth28: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:02:c9:e6:03:11 brd ff:ff:ff:ff:ff:ff
> 22: eth27: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT qlen 1000
> link/ether 00:02:c9:e6:03:12 brd ff:ff:ff:ff:ff:ff
>
> # pwd
>
> /sys/class/net
>
> # ls -l
> total 0
> lrwxrwxrwx 1 root root 0 Apr 24 18:44 br0 -> ../../devices/virtual/net/br0
> lrwxrwxrwx 1 root root 0 Apr 24 18:44 eth0 ->
> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.0/net/eth0
> lrwxrwxrwx 1 root root 0 Apr 24 18:44 eth1 ->
> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.1/net/eth1
> lrwxrwxrwx 1 root root 0 Apr 24 18:45 eth10 ->
> ../../devices/pci0000:00/0000:00:09.0/0000:05:00.1/net/eth10
> lrwxrwxrwx 1 root root 0 Apr 24 18:45 eth11 ->
> ../../devices/pci0000:00/0000:00:09.0/0000:05:00.0/net/eth11
> lrwxrwxrwx 1 root root 0 Apr 24 18:51 eth16 ->
> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth16
> lrwxrwxrwx 1 root root 0 Apr 24 18:51 eth17 ->
> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth17
> lrwxrwxrwx 1 root root 0 Apr 24 18:51 eth18 ->
> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth18
> lrwxrwxrwx 1 root root 0 Apr 24 18:51 eth19 ->
> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth19
> lrwxrwxrwx 1 root root 0 Apr 24 18:44 eth20 ->
> ../../devices/pci0000:00/0000:00:09.0/0000:05:10.1/net/eth20
> lrwxrwxrwx 1 root root 0 Apr 24 18:44 eth22 ->
> ../../devices/pci0000:00/0000:00:09.0/0000:05:10.0/net/eth22
> lrwxrwxrwx 1 root root 0 Apr 24 18:51 eth27 ->
> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.2/net/eth27
> lrwxrwxrwx 1 root root 0 Apr 24 18:51 eth28 ->
> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.2/net/eth28
> lrwxrwxrwx 1 root root 0 Apr 24 18:44 lo -> ../../devices/virtual/net/lo
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: getting VF link info seems to be broken in 3.9-rc8
2013-04-24 16:08 ` Li, Sibai
@ 2013-04-24 16:34 ` Or Gerlitz
2013-04-24 16:44 ` Li, Sibai
0 siblings, 1 reply; 20+ messages in thread
From: Or Gerlitz @ 2013-04-24 16:34 UTC (permalink / raw)
To: Li, Sibai
Cc: Rose, Gregory V, David Miller, Duyck, Alexander H, netdev,
Rony Efraim
On 24/04/2013 19:08, Li, Sibai wrote:
> The vfs are listed, eth22/eth20. Just the mac address is different. Reload the vf driver to see if it corrects the thing.
Sure, the VF are listed as NICs of their own, what's isn't listed is the
VF info from the PF NIC link info, e.g here's the casewhere its working,
on kernel 3.8.8-- my problem isn't the MAC misalignment
8: eth11: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state
DOWN mode DEFAULT qlen 1000
link/ether 00:1b:21:55:1d:00 brd ff:ff:ff:ff:ff:ff
vf 0 MAC 00:11:00:11:00:11, spoof checking on
9: eth10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
mode DORMANT qlen 1000
link/ether 00:1b:21:55:1d:01 brd ff:ff:ff:ff:ff:ff
vf 0 MAC 00:10:00:10:00:10, spoof checking on
^ permalink raw reply [flat|nested] 20+ messages in thread
* RE: getting VF link info seems to be broken in 3.9-rc8
2013-04-24 16:34 ` Or Gerlitz
@ 2013-04-24 16:44 ` Li, Sibai
2013-04-24 16:45 ` Or Gerlitz
2013-04-24 16:48 ` Rose, Gregory V
0 siblings, 2 replies; 20+ messages in thread
From: Li, Sibai @ 2013-04-24 16:44 UTC (permalink / raw)
To: Or Gerlitz
Cc: Rose, Gregory V, David Miller, Duyck, Alexander H, netdev,
Rony Efraim
This means that the VF driver is loaded before the PF is loaded. Did you reload the VF after the PF is loaded?
> -----Original Message-----
> From: Or Gerlitz [mailto:ogerlitz@mellanox.com]
> Sent: Wednesday, April 24, 2013 9:35 AM
> To: Li, Sibai
> Cc: Rose, Gregory V; David Miller; Duyck, Alexander H; netdev; Rony Efraim
> Subject: Re: getting VF link info seems to be broken in 3.9-rc8
>
> On 24/04/2013 19:08, Li, Sibai wrote:
> > The vfs are listed, eth22/eth20. Just the mac address is different. Reload the
> vf driver to see if it corrects the thing.
>
> Sure, the VF are listed as NICs of their own, what's isn't listed is the VF info from
> the PF NIC link info, e.g here's the casewhere its working, on kernel 3.8.8-- my
> problem isn't the MAC misalignment
>
> 8: eth11: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state
> DOWN mode DEFAULT qlen 1000
> link/ether 00:1b:21:55:1d:00 brd ff:ff:ff:ff:ff:ff
> vf 0 MAC 00:11:00:11:00:11, spoof checking on
> 9: eth10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state
> UP mode DORMANT qlen 1000
> link/ether 00:1b:21:55:1d:01 brd ff:ff:ff:ff:ff:ff
> vf 0 MAC 00:10:00:10:00:10, spoof checking on
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: getting VF link info seems to be broken in 3.9-rc8
2013-04-24 16:44 ` Li, Sibai
@ 2013-04-24 16:45 ` Or Gerlitz
2013-04-24 16:48 ` Rose, Gregory V
1 sibling, 0 replies; 20+ messages in thread
From: Or Gerlitz @ 2013-04-24 16:45 UTC (permalink / raw)
To: Li, Sibai
Cc: Rose, Gregory V, David Miller, Duyck, Alexander H, netdev,
Rony Efraim
On 24/04/2013 19:44, Li, Sibai wrote:
> This means that the VF driver is loaded before the PF is loaded. Did you reload the VF after the PF is loaded?
I have reloaded the VF driver now and it didn't make any difference
^ permalink raw reply [flat|nested] 20+ messages in thread
* RE: getting VF link info seems to be broken in 3.9-rc8
2013-04-24 16:44 ` Li, Sibai
2013-04-24 16:45 ` Or Gerlitz
@ 2013-04-24 16:48 ` Rose, Gregory V
2013-04-24 16:56 ` Li, Sibai
1 sibling, 1 reply; 20+ messages in thread
From: Rose, Gregory V @ 2013-04-24 16:48 UTC (permalink / raw)
To: Li, Sibai, Or Gerlitz
Cc: David Miller, Duyck, Alexander H, netdev, Rony Efraim
> -----Original Message-----
> From: Li, Sibai
> Sent: Wednesday, April 24, 2013 9:44 AM
> To: Or Gerlitz
> Cc: Rose, Gregory V; David Miller; Duyck, Alexander H; netdev; Rony Efraim
> Subject: RE: getting VF link info seems to be broken in 3.9-rc8
>
> This means that the VF driver is loaded before the PF is loaded. Did you
> reload the VF after the PF is loaded?
Sibai,
It should not matter whether the VF driver is loaded or not when listing the VFs using "ip link show".
- Greg
> > -----Original Message-----
> > From: Or Gerlitz [mailto:ogerlitz@mellanox.com]
> > Sent: Wednesday, April 24, 2013 9:35 AM
> > To: Li, Sibai
> > Cc: Rose, Gregory V; David Miller; Duyck, Alexander H; netdev; Rony
> > Efraim
> > Subject: Re: getting VF link info seems to be broken in 3.9-rc8
> >
> > On 24/04/2013 19:08, Li, Sibai wrote:
> > > The vfs are listed, eth22/eth20. Just the mac address is different.
> > > Reload the
> > vf driver to see if it corrects the thing.
> >
> > Sure, the VF are listed as NICs of their own, what's isn't listed is
> > the VF info from the PF NIC link info, e.g here's the casewhere its
> > working, on kernel 3.8.8-- my problem isn't the MAC misalignment
> >
> > 8: eth11: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state
> > DOWN mode DEFAULT qlen 1000
> > link/ether 00:1b:21:55:1d:00 brd ff:ff:ff:ff:ff:ff
> > vf 0 MAC 00:11:00:11:00:11, spoof checking on
> > 9: eth10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
> > mode DORMANT qlen 1000
> > link/ether 00:1b:21:55:1d:01 brd ff:ff:ff:ff:ff:ff
> > vf 0 MAC 00:10:00:10:00:10, spoof checking on
^ permalink raw reply [flat|nested] 20+ messages in thread
* RE: getting VF link info seems to be broken in 3.9-rc8
2013-04-24 16:48 ` Rose, Gregory V
@ 2013-04-24 16:56 ` Li, Sibai
0 siblings, 0 replies; 20+ messages in thread
From: Li, Sibai @ 2013-04-24 16:56 UTC (permalink / raw)
To: Rose, Gregory V, Or Gerlitz
Cc: David Miller, Duyck, Alexander H, netdev, Rony Efraim
Ok I see what you sew now, RTNETLINK might be broken.
> -----Original Message-----
> From: Rose, Gregory V
> Sent: Wednesday, April 24, 2013 9:49 AM
> To: Li, Sibai; Or Gerlitz
> Cc: David Miller; Duyck, Alexander H; netdev; Rony Efraim
> Subject: RE: getting VF link info seems to be broken in 3.9-rc8
>
>
>
> > -----Original Message-----
> > From: Li, Sibai
> > Sent: Wednesday, April 24, 2013 9:44 AM
> > To: Or Gerlitz
> > Cc: Rose, Gregory V; David Miller; Duyck, Alexander H; netdev; Rony
> > Efraim
> > Subject: RE: getting VF link info seems to be broken in 3.9-rc8
> >
> > This means that the VF driver is loaded before the PF is loaded. Did
> > you reload the VF after the PF is loaded?
>
> Sibai,
>
> It should not matter whether the VF driver is loaded or not when listing the VFs
> using "ip link show".
>
> - Greg
>
> > > -----Original Message-----
> > > From: Or Gerlitz [mailto:ogerlitz@mellanox.com]
> > > Sent: Wednesday, April 24, 2013 9:35 AM
> > > To: Li, Sibai
> > > Cc: Rose, Gregory V; David Miller; Duyck, Alexander H; netdev; Rony
> > > Efraim
> > > Subject: Re: getting VF link info seems to be broken in 3.9-rc8
> > >
> > > On 24/04/2013 19:08, Li, Sibai wrote:
> > > > The vfs are listed, eth22/eth20. Just the mac address is different.
> > > > Reload the
> > > vf driver to see if it corrects the thing.
> > >
> > > Sure, the VF are listed as NICs of their own, what's isn't listed is
> > > the VF info from the PF NIC link info, e.g here's the casewhere its
> > > working, on kernel 3.8.8-- my problem isn't the MAC misalignment
> > >
> > > 8: eth11: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq
> > > state DOWN mode DEFAULT qlen 1000
> > > link/ether 00:1b:21:55:1d:00 brd ff:ff:ff:ff:ff:ff
> > > vf 0 MAC 00:11:00:11:00:11, spoof checking on
> > > 9: eth10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq
> state
> > > UP mode DORMANT qlen 1000
> > > link/ether 00:1b:21:55:1d:01 brd ff:ff:ff:ff:ff:ff
> > > vf 0 MAC 00:10:00:10:00:10, spoof checking on
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: getting VF link info seems to be broken in 3.9-rc8
2013-04-24 15:05 ` Williams, Mitch A
@ 2013-04-25 18:29 ` Alexander Duyck
2013-04-25 19:24 ` David Miller
0 siblings, 1 reply; 20+ messages in thread
From: Alexander Duyck @ 2013-04-25 18:29 UTC (permalink / raw)
To: Williams, Mitch A
Cc: Or Gerlitz, David Miller, Rose, Gregory V,
e1000-devel@lists.sourceforge.net, netdev, Rony Efraim,
Stephen Hemminger
I did a bit of digging and it looks like the issue is that the
ext_filter_mask is not being found in the message received in
rtnl_dump_ifinfo. I'm still trying to figure out why the kernel isn't
finding the flag when it was finding it previously, but I'm not much of
a netlink expert.
Thanks,
Alex
On 04/24/2013 08:05 AM, Williams, Mitch A wrote:
> Adding the e1000-devel list.
>
>> -----Original Message-----
>> From: netdev-owner@vger.kernel.org [mailto:netdev-owner@vger.kernel.org] On
>> Behalf Of Or Gerlitz
>> Sent: Wednesday, April 24, 2013 6:08 AM
>> To: David Miller; Rose, Gregory V; Duyck, Alexander H
>> Cc: netdev; Rony Efraim
>> Subject: getting VF link info seems to be broken in 3.9-rc8
>>
>> [re-posting as of non text only content in the 1st post - sorry for the
>> spam] Rony Efraim <ronye@mellanox.com>observed a regression in 3.9-rc8
>> with respect to getting VFs link info from user space. Using latest
>> iproute2 we can easily get the VF info on 3.8.8 but not on the net git
>> which is now aligned on 3.9-rc8, will try to bisect it later
>> today/tomorrow, but for the sake of maybe someone has an insight of the
>> bug, reporting it now...
>>
>> You can see below that for eth10 and eth11 which are ixgbe PFs, the VF
>> info is reported on 3.8.8 but not on 3.9-rc8, we see the same problem
>> with patches to mlx4 which implement ndo_get_vf_info which gives some
>> evidence that the problem isn't in the ixgbe driver, but rather
>> somewhere else, maybe the networking core, from quick looking on
>> net/core/rtnetlink.c I don't see there a smoking gun, anyway
>>
>> net.git]# git log --oneline v3.8..v3.9-rc8 net/core/rtnetlink.c
>> 88c5b5c rtnetlink: Call nlmsg_parse() with correct header length
>> fcca143 rtnetlink: fix error return code in rtnl_link_fill()
>> a5b8db9 rtnetlink: Mask the rta_type when range checking
>> 84d73cd rtnl: fix info leak on RTM_GETLINK request for VF devices
>> b67bfe0 hlist: drop the node parameter from iterators
>> 1690be6 bridge: Add vlan support to static neighbors
>> 6cbdcee bridge: Dump vlan information from a bridge port
>> 407af32 bridge: Add netlink interface to configure vlans on bridge ports
>> c5c3510 netns: fdb: allow unprivileged users to add/del fdb entries
>> 2afb9b5 ethtool: set addr_assign_type to NET_ADDR_SET when addr is
>> passed on create
>> 471cb5a bonding: remove usage of dev->master
>> 898e506 rtnetlink: remove usage of dev->master
>> e7c3273 rtnl: use dev_set_mac_address() instead of plain ndo_
>> 9a57247 rtnl: expose carrier value with possibility to set it
>>
>>
>> Or.
>>
>> --> using 3.8.8
>>
>> # lspci | grep Eth
>> 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
>> Connection (rev 01)
>> 01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network
>> Connection (rev 01)
>> 05:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
>> SFI/SFP+ Network Connection (rev 01)
>> 05:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
>> SFI/SFP+ Network Connection (rev 01)
>> 05:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
>> Virtual Function (rev 01)
>> 05:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller
>> Virtual Function (rev 01)
>>
>> # cd /sys/class/net/
>> # ls -l
>> lrwxrwxrwx 1 root root 0 Apr 24 12:17 br0 -> ../../devices/virtual/net/br0
>> lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth0 ->
>> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.0/net/eth0
>> lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth1 ->
>> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.1/net/eth1
>> lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth10 ->
>> ../../devices/pci0000:00/0000:00:09.0/0000:05:00.1/net/eth10
>> lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth11 ->
>> ../../devices/pci0000:00/0000:00:09.0/0000:05:00.0/net/eth11
>> lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth16 ->
>> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth16
>> lrwxrwxrwx 1 root root 0 Apr 24 12:18 eth17 ->
>> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth17
>> lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth18 ->
>> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth18
>> lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth19 ->
>> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth19
>> lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth20 ->
>> ../../devices/pci0000:00/0000:00:09.0/0000:05:10.1/net/eth20
>> lrwxrwxrwx 1 root root 0 Apr 24 12:17 eth22 ->
>> ../../devices/pci0000:00/0000:00:09.0/0000:05:10.0/net/eth22
>> lrwxrwxrwx 1 root root 0 Apr 24 12:17 lo -> ../../devices/virtual/net/lo
>>
>>
>>
>> # modprobe -r ixgbevf
>> # ip link set dev eth10 vf 0 mac 00:10:00:10:00:10
>> # ip link set dev eth11 vf 0 mac 00:11:00:11:00:11
>> # modprobe ixgbevf
>> # ip link show
>> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode
>> DEFAULT
>> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0
>> state UP mode DEFAULT qlen 1000
>> link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
>> 3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
>> DEFAULT qlen 1000
>> link/ether 00:25:90:36:9c:ed brd ff:ff:ff:ff:ff:ff
>> 4: eth16: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
>> DEFAULT qlen 1000
>> link/ether 00:02:c9:e6:01:11 brd ff:ff:ff:ff:ff:ff
>> 5: eth17: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
>> DEFAULT qlen 1000
>> link/ether 00:02:c9:e6:01:12 brd ff:ff:ff:ff:ff:ff
>> 6: eth18: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
>> DEFAULT qlen 1000
>> link/ether 00:02:c9:e6:02:11 brd ff:ff:ff:ff:ff:ff
>> 7: eth19: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
>> DEFAULT qlen 1000
>> link/ether 00:02:c9:e6:02:12 brd ff:ff:ff:ff:ff:ff
>> 8: eth11: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state
>> DOWN mode DEFAULT qlen 1000
>> link/ether 00:1b:21:55:1d:00 brd ff:ff:ff:ff:ff:ff
>> vf 0 MAC 00:11:00:11:00:11, spoof checking on
>> 9: eth10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
>> mode DORMANT qlen 1000
>> link/ether 00:1b:21:55:1d:01 brd ff:ff:ff:ff:ff:ff
>> vf 0 MAC 00:10:00:10:00:10, spoof checking on
>> 12: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
>> UP mode DEFAULT
>> link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
>> 23: eth22: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
>> DEFAULT qlen 1000
>> link/ether 00:11:00:11:00:11 brd ff:ff:ff:ff:ff:ff
>> 24: eth20: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
>> DEFAULT qlen 1000
>> link/ether 00:10:00:10:00:10 brd ff:ff:ff:ff:ff:ff
>>
>>
>> --> using 3.9-rc8
>>
>> # lspci | grep Eth
>> 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
>> Connection (rev 01)
>> 01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network
>> Connection (rev 01)
>> 05:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
>> SFI/SFP+ Network Connection (rev 01)
>> 05:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
>> SFI/SFP+ Network Connection (rev 01)
>> 05:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
>> Virtual Function (rev 01)
>> 05:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller
>> Virtual Function (rev 01)
>>
>> # cd /sys/class/net/
>> # ls -l
>> lrwxrwxrwx 1 root root 0 Apr 24 11:48 br0 -> ../../devices/virtual/net/br0
>> lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth0 ->
>> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.0/net/eth0
>> lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth1 ->
>> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.1/net/eth1
>> lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth10 ->
>> ../../devices/pci0000:00/0000:00:09.0/0000:05:00.1/net/eth10
>> lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth11 ->
>> ../../devices/pci0000:00/0000:00:09.0/0000:05:00.0/net/eth11
>> lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth16 ->
>> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth16
>> lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth17 ->
>> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth17
>> lrwxrwxrwx 1 root root 0 Apr 24 11:49 eth18 ->
>> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth18
>> lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth19 ->
>> ../../devices/pci0000:00/0000:00:07.0/0000:04:00.1/net/eth19
>> lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth20 ->
>> ../../devices/pci0000:00/0000:00:09.0/0000:05:10.1/net/eth20
>> lrwxrwxrwx 1 root root 0 Apr 24 11:48 eth22 ->
>> ../../devices/pci0000:00/0000:00:09.0/0000:05:10.0/net/eth22
>> lrwxrwxrwx 1 root root 0 Apr 24 11:47 lo -> ../../devices/virtual/net/lo
>>
>> # ip link show <--- no sign for VFs
>> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode
>> DEFAULT
>> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0
>> state UP mode DEFAULT qlen 1000
>> link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
>> 3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
>> DEFAULT qlen 1000
>> link/ether 00:25:90:36:9c:ed brd ff:ff:ff:ff:ff:ff
>> 4: eth11: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
>> DEFAULT qlen 1000
>> link/ether 00:1b:21:55:1d:00 brd ff:ff:ff:ff:ff:ff
>> 5: eth10: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
>> DEFAULT qlen 1000
>> link/ether 00:1b:21:55:1d:01 brd ff:ff:ff:ff:ff:ff
>> 6: eth16: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
>> DEFAULT qlen 1000
>> link/ether 00:02:c9:e6:01:11 brd ff:ff:ff:ff:ff:ff
>> 7: eth17: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
>> DEFAULT qlen 1000
>> link/ether 00:02:c9:e6:01:12 brd ff:ff:ff:ff:ff:ff
>> 8: eth18: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
>> DEFAULT qlen 1000
>> link/ether 00:02:c9:e6:02:11 brd ff:ff:ff:ff:ff:ff
>> 9: eth19: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
>> DEFAULT qlen 1000
>> link/ether 00:02:c9:e6:02:12 brd ff:ff:ff:ff:ff:ff
>> 10: eth22: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
>> DEFAULT qlen 1000
>> link/ether 3e:9c:76:28:d9:da brd ff:ff:ff:ff:ff:ff
>> 11: eth20: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
>> DEFAULT qlen 1000
>> link/ether ea:f6:42:a7:d9:40 brd ff:ff:ff:ff:ff:ff
>> 12: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
>> UP mode DEFAULT
>> link/ether 00:25:90:36:9c:ec brd ff:ff:ff:ff:ff:ff
>> --
>> To unsubscribe from this list: send the line "unsubscribe netdev" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: getting VF link info seems to be broken in 3.9-rc8
2013-04-25 18:29 ` Alexander Duyck
@ 2013-04-25 19:24 ` David Miller
2013-04-25 20:20 ` Alexander Duyck
0 siblings, 1 reply; 20+ messages in thread
From: David Miller @ 2013-04-25 19:24 UTC (permalink / raw)
To: alexander.h.duyck
Cc: mitch.a.williams, ogerlitz, gregory.v.rose, e1000-devel, netdev,
ronye, shemminger
From: Alexander Duyck <alexander.h.duyck@intel.com>
Date: Thu, 25 Apr 2013 11:29:04 -0700
> I did a bit of digging and it looks like the issue is that the
> ext_filter_mask is not being found in the message received in
> rtnl_dump_ifinfo. I'm still trying to figure out why the kernel isn't
> finding the flag when it was finding it previously, but I'm not much of
> a netlink expert.
I wonder if this has something to do with this commit:
====================
>From 88c5b5ce5cb57af6ca2a7cf4d5715fa320448ff9 Mon Sep 17 00:00:00 2001
From: Michael Riesch <michael.riesch@omicron.at>
Date: Mon, 8 Apr 2013 05:45:26 +0000
Subject: [PATCH] rtnetlink: Call nlmsg_parse() with correct header length
Signed-off-by: Michael Riesch <michael.riesch@omicron.at>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Jiri Benc <jbenc@redhat.com>
Cc: "Theodore Ts'o" <tytso@mit.edu>
Cc: linux-kernel@vger.kernel.org
Acked-by: Mark Rustad <mark.d.rustad@intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
---
net/core/rtnetlink.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c
index b65441d..23854b5 100644
--- a/net/core/rtnetlink.c
+++ b/net/core/rtnetlink.c
@@ -1072,7 +1072,7 @@ static int rtnl_dump_ifinfo(struct sk_buff *skb, struct netlink_callback *cb)
rcu_read_lock();
cb->seq = net->dev_base_seq;
- if (nlmsg_parse(cb->nlh, sizeof(struct rtgenmsg), tb, IFLA_MAX,
+ if (nlmsg_parse(cb->nlh, sizeof(struct ifinfomsg), tb, IFLA_MAX,
ifla_policy) >= 0) {
if (tb[IFLA_EXT_MASK])
@@ -1922,7 +1922,7 @@ static u16 rtnl_calcit(struct sk_buff *skb, struct nlmsghdr *nlh)
u32 ext_filter_mask = 0;
u16 min_ifinfo_dump_size = 0;
- if (nlmsg_parse(nlh, sizeof(struct rtgenmsg), tb, IFLA_MAX,
+ if (nlmsg_parse(nlh, sizeof(struct ifinfomsg), tb, IFLA_MAX,
ifla_policy) >= 0) {
if (tb[IFLA_EXT_MASK])
ext_filter_mask = nla_get_u32(tb[IFLA_EXT_MASK]);
--
1.7.11.7
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: getting VF link info seems to be broken in 3.9-rc8
2013-04-25 19:24 ` David Miller
@ 2013-04-25 20:20 ` Alexander Duyck
2013-04-25 20:25 ` David Miller
0 siblings, 1 reply; 20+ messages in thread
From: Alexander Duyck @ 2013-04-25 20:20 UTC (permalink / raw)
To: David Miller
Cc: mitch.a.williams, ogerlitz, gregory.v.rose, e1000-devel, netdev,
ronye, shemminger
On 04/25/2013 12:24 PM, David Miller wrote:
> From: Alexander Duyck <alexander.h.duyck@intel.com>
> Date: Thu, 25 Apr 2013 11:29:04 -0700
>
>> I did a bit of digging and it looks like the issue is that the
>> ext_filter_mask is not being found in the message received in
>> rtnl_dump_ifinfo. I'm still trying to figure out why the kernel isn't
>> finding the flag when it was finding it previously, but I'm not much of
>> a netlink expert.
> I wonder if this has something to do with this commit:
>
> ====================
> From 88c5b5ce5cb57af6ca2a7cf4d5715fa320448ff9 Mon Sep 17 00:00:00 2001
> From: Michael Riesch <michael.riesch@omicron.at>
> Date: Mon, 8 Apr 2013 05:45:26 +0000
> Subject: [PATCH] rtnetlink: Call nlmsg_parse() with correct header length
>
> Signed-off-by: Michael Riesch <michael.riesch@omicron.at>
> Cc: "David S. Miller" <davem@davemloft.net>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Jiri Benc <jbenc@redhat.com>
> Cc: "Theodore Ts'o" <tytso@mit.edu>
> Cc: linux-kernel@vger.kernel.org
> Acked-by: Mark Rustad <mark.d.rustad@intel.com>
> Signed-off-by: David S. Miller <davem@davemloft.net>
> ---
> net/core/rtnetlink.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c
> index b65441d..23854b5 100644
> --- a/net/core/rtnetlink.c
> +++ b/net/core/rtnetlink.c
> @@ -1072,7 +1072,7 @@ static int rtnl_dump_ifinfo(struct sk_buff *skb, struct netlink_callback *cb)
> rcu_read_lock();
> cb->seq = net->dev_base_seq;
>
> - if (nlmsg_parse(cb->nlh, sizeof(struct rtgenmsg), tb, IFLA_MAX,
> + if (nlmsg_parse(cb->nlh, sizeof(struct ifinfomsg), tb, IFLA_MAX,
> ifla_policy) >= 0) {
>
> if (tb[IFLA_EXT_MASK])
> @@ -1922,7 +1922,7 @@ static u16 rtnl_calcit(struct sk_buff *skb, struct nlmsghdr *nlh)
> u32 ext_filter_mask = 0;
> u16 min_ifinfo_dump_size = 0;
>
> - if (nlmsg_parse(nlh, sizeof(struct rtgenmsg), tb, IFLA_MAX,
> + if (nlmsg_parse(nlh, sizeof(struct ifinfomsg), tb, IFLA_MAX,
> ifla_policy) >= 0) {
> if (tb[IFLA_EXT_MASK])
> ext_filter_mask = nla_get_u32(tb[IFLA_EXT_MASK]);
I thought that as well. I tried reverting it and the issue is still there.
However, I do think this may be part of the issue since I added a printk
to dump nlmsg_attrlen before going into the nlmsg_parse and with
ifinfomsg the attrlen is -12, with rtgenmsg it is 0.
Thanks,
Alex
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: getting VF link info seems to be broken in 3.9-rc8
2013-04-25 20:20 ` Alexander Duyck
@ 2013-04-25 20:25 ` David Miller
2013-04-25 20:36 ` Alexander Duyck
0 siblings, 1 reply; 20+ messages in thread
From: David Miller @ 2013-04-25 20:25 UTC (permalink / raw)
To: alexander.h.duyck
Cc: mitch.a.williams, ogerlitz, gregory.v.rose, e1000-devel, netdev,
ronye, shemminger
From: Alexander Duyck <alexander.h.duyck@intel.com>
Date: Thu, 25 Apr 2013 13:20:24 -0700
> On 04/25/2013 12:24 PM, David Miller wrote:
>> From: Alexander Duyck <alexander.h.duyck@intel.com>
>> Date: Thu, 25 Apr 2013 11:29:04 -0700
>>
>> diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c
>> index b65441d..23854b5 100644
>> --- a/net/core/rtnetlink.c
>> +++ b/net/core/rtnetlink.c
>> @@ -1072,7 +1072,7 @@ static int rtnl_dump_ifinfo(struct sk_buff *skb, struct netlink_callback *cb)
>> rcu_read_lock();
>> cb->seq = net->dev_base_seq;
>>
>> - if (nlmsg_parse(cb->nlh, sizeof(struct rtgenmsg), tb, IFLA_MAX,
>> + if (nlmsg_parse(cb->nlh, sizeof(struct ifinfomsg), tb, IFLA_MAX,
>> ifla_policy) >= 0) {
>>
>> if (tb[IFLA_EXT_MASK])
>> @@ -1922,7 +1922,7 @@ static u16 rtnl_calcit(struct sk_buff *skb, struct nlmsghdr *nlh)
>> u32 ext_filter_mask = 0;
>> u16 min_ifinfo_dump_size = 0;
>>
>> - if (nlmsg_parse(nlh, sizeof(struct rtgenmsg), tb, IFLA_MAX,
>> + if (nlmsg_parse(nlh, sizeof(struct ifinfomsg), tb, IFLA_MAX,
>> ifla_policy) >= 0) {
>> if (tb[IFLA_EXT_MASK])
>> ext_filter_mask = nla_get_u32(tb[IFLA_EXT_MASK]);
>
> I thought that as well. I tried reverting it and the issue is still there.
>
> However, I do think this may be part of the issue since I added a printk
> to dump nlmsg_attrlen before going into the nlmsg_parse and with
> ifinfomsg the attrlen is -12, with rtgenmsg it is 0.
I wonder if we are seeing two ways tools are making these calls, some are
passing rtgenmsg and some are passing ifinfomsg. The latter, I am mostly
convinced, is what we must see here from properly written applications.
That would be really unfortunate, but seeing a nlmsg_attrlen() of -12 would
seem to confirm that a rtgenmsg was used.
I guess you're using iproute2?
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: getting VF link info seems to be broken in 3.9-rc8
2013-04-25 20:25 ` David Miller
@ 2013-04-25 20:36 ` Alexander Duyck
2013-04-25 20:45 ` Stephen Hemminger
0 siblings, 1 reply; 20+ messages in thread
From: Alexander Duyck @ 2013-04-25 20:36 UTC (permalink / raw)
To: David Miller
Cc: mitch.a.williams, ogerlitz, gregory.v.rose, e1000-devel, netdev,
ronye, shemminger
On 04/25/2013 01:25 PM, David Miller wrote:
> From: Alexander Duyck <alexander.h.duyck@intel.com>
> Date: Thu, 25 Apr 2013 13:20:24 -0700
>
>> On 04/25/2013 12:24 PM, David Miller wrote:
>>> From: Alexander Duyck <alexander.h.duyck@intel.com>
>>> Date: Thu, 25 Apr 2013 11:29:04 -0700
>>>
>>> diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c
>>> index b65441d..23854b5 100644
>>> --- a/net/core/rtnetlink.c
>>> +++ b/net/core/rtnetlink.c
>>> @@ -1072,7 +1072,7 @@ static int rtnl_dump_ifinfo(struct sk_buff *skb, struct netlink_callback *cb)
>>> rcu_read_lock();
>>> cb->seq = net->dev_base_seq;
>>>
>>> - if (nlmsg_parse(cb->nlh, sizeof(struct rtgenmsg), tb, IFLA_MAX,
>>> + if (nlmsg_parse(cb->nlh, sizeof(struct ifinfomsg), tb, IFLA_MAX,
>>> ifla_policy) >= 0) {
>>>
>>> if (tb[IFLA_EXT_MASK])
>>> @@ -1922,7 +1922,7 @@ static u16 rtnl_calcit(struct sk_buff *skb, struct nlmsghdr *nlh)
>>> u32 ext_filter_mask = 0;
>>> u16 min_ifinfo_dump_size = 0;
>>>
>>> - if (nlmsg_parse(nlh, sizeof(struct rtgenmsg), tb, IFLA_MAX,
>>> + if (nlmsg_parse(nlh, sizeof(struct ifinfomsg), tb, IFLA_MAX,
>>> ifla_policy) >= 0) {
>>> if (tb[IFLA_EXT_MASK])
>>> ext_filter_mask = nla_get_u32(tb[IFLA_EXT_MASK]);
>> I thought that as well. I tried reverting it and the issue is still there.
>>
>> However, I do think this may be part of the issue since I added a printk
>> to dump nlmsg_attrlen before going into the nlmsg_parse and with
>> ifinfomsg the attrlen is -12, with rtgenmsg it is 0.
> I wonder if we are seeing two ways tools are making these calls, some are
> passing rtgenmsg and some are passing ifinfomsg. The latter, I am mostly
> convinced, is what we must see here from properly written applications.
>
> That would be really unfortunate, but seeing a nlmsg_attrlen() of -12 would
> seem to confirm that a rtgenmsg was used.
>
> I guess you're using iproute2?
Yes. All I am doing is ip link show and previously this would display
VF info as well as PF. Now the VF info is not there.
>From what I can tell the problem call is rtnl_wilddump_req_filter since
it is only passing a rtgenmsg and asking us to dump the link with the
RTEXT_FILTER_VF filter mask.
Thanks,
Alex
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: getting VF link info seems to be broken in 3.9-rc8
2013-04-25 20:36 ` Alexander Duyck
@ 2013-04-25 20:45 ` Stephen Hemminger
2013-04-25 20:49 ` David Miller
0 siblings, 1 reply; 20+ messages in thread
From: Stephen Hemminger @ 2013-04-25 20:45 UTC (permalink / raw)
To: Alexander Duyck
Cc: e1000-devel, netdev, shemminger, ronye, ogerlitz, David Miller
On Thu, 25 Apr 2013 13:36:06 -0700
Alexander Duyck <alexander.h.duyck@intel.com> wrote:
> On 04/25/2013 01:25 PM, David Miller wrote:
> > From: Alexander Duyck <alexander.h.duyck@intel.com>
> > Date: Thu, 25 Apr 2013 13:20:24 -0700
> >
> >> On 04/25/2013 12:24 PM, David Miller wrote:
> >>> From: Alexander Duyck <alexander.h.duyck@intel.com>
> >>> Date: Thu, 25 Apr 2013 11:29:04 -0700
> >>>
> >>> diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c
> >>> index b65441d..23854b5 100644
> >>> --- a/net/core/rtnetlink.c
> >>> +++ b/net/core/rtnetlink.c
> >>> @@ -1072,7 +1072,7 @@ static int rtnl_dump_ifinfo(struct sk_buff *skb, struct netlink_callback *cb)
> >>> rcu_read_lock();
> >>> cb->seq = net->dev_base_seq;
> >>>
> >>> - if (nlmsg_parse(cb->nlh, sizeof(struct rtgenmsg), tb, IFLA_MAX,
> >>> + if (nlmsg_parse(cb->nlh, sizeof(struct ifinfomsg), tb, IFLA_MAX,
> >>> ifla_policy) >= 0) {
> >>>
> >>> if (tb[IFLA_EXT_MASK])
> >>> @@ -1922,7 +1922,7 @@ static u16 rtnl_calcit(struct sk_buff *skb, struct nlmsghdr *nlh)
> >>> u32 ext_filter_mask = 0;
> >>> u16 min_ifinfo_dump_size = 0;
> >>>
> >>> - if (nlmsg_parse(nlh, sizeof(struct rtgenmsg), tb, IFLA_MAX,
> >>> + if (nlmsg_parse(nlh, sizeof(struct ifinfomsg), tb, IFLA_MAX,
> >>> ifla_policy) >= 0) {
> >>> if (tb[IFLA_EXT_MASK])
> >>> ext_filter_mask = nla_get_u32(tb[IFLA_EXT_MASK]);
> >> I thought that as well. I tried reverting it and the issue is still there.
> >>
> >> However, I do think this may be part of the issue since I added a printk
> >> to dump nlmsg_attrlen before going into the nlmsg_parse and with
> >> ifinfomsg the attrlen is -12, with rtgenmsg it is 0.
> > I wonder if we are seeing two ways tools are making these calls, some are
> > passing rtgenmsg and some are passing ifinfomsg. The latter, I am mostly
> > convinced, is what we must see here from properly written applications.
> >
> > That would be really unfortunate, but seeing a nlmsg_attrlen() of -12 would
> > seem to confirm that a rtgenmsg was used.
> >
> > I guess you're using iproute2?
>
> Yes. All I am doing is ip link show and previously this would display
> VF info as well as PF. Now the VF info is not there.
>
> From what I can tell the problem call is rtnl_wilddump_req_filter since
> it is only passing a rtgenmsg and asking us to dump the link with the
> RTEXT_FILTER_VF filter mask.
It looks like a bug in the initial code for filter. in this case, it calls:
ip_list_flush_or_save
rtnl_wilddump_req_filter
which sends 'struct rtgenmsg'
This was probably a mistake in the original flags addition, not sure if we can
fix it now that ABI is set. Probably have to revert the kernel change.
commit bd886ebb1ffd84301caa2341b671df9a9e2db4c9
Author: Rose, Gregory V <gregory.v.rose@intel.com>
Date: Tue Feb 21 10:43:09 2012 +0000
iproute2: Add netlink attribute to filter dump requests
Add a new netlink attribute type to the dump request to allow
filtering of the information returned for the respective matching
interfaces. At this time the only filter defined is to request
virtual function (VF) device info for interfaces that attached VFs.
It will also be possible to extend the request with other yet to be
defined netlink attributes in the future.
Signed-off-by: Greg Rose <gregory.v.rose@intel.com>
------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: getting VF link info seems to be broken in 3.9-rc8
2013-04-25 20:45 ` Stephen Hemminger
@ 2013-04-25 20:49 ` David Miller
2013-04-25 21:10 ` Alexander Duyck
0 siblings, 1 reply; 20+ messages in thread
From: David Miller @ 2013-04-25 20:49 UTC (permalink / raw)
To: stephen
Cc: alexander.h.duyck, mitch.a.williams, ogerlitz, gregory.v.rose,
e1000-devel, netdev, ronye, shemminger
From: Stephen Hemminger <stephen@networkplumber.org>
Date: Thu, 25 Apr 2013 13:45:13 -0700
> On Thu, 25 Apr 2013 13:36:06 -0700
> Alexander Duyck <alexander.h.duyck@intel.com> wrote:
>
>> On 04/25/2013 01:25 PM, David Miller wrote:
>> > From: Alexander Duyck <alexander.h.duyck@intel.com>
>> > Date: Thu, 25 Apr 2013 13:20:24 -0700
>> >
>> >> On 04/25/2013 12:24 PM, David Miller wrote:
>> >>> From: Alexander Duyck <alexander.h.duyck@intel.com>
>> >>> Date: Thu, 25 Apr 2013 11:29:04 -0700
>> >>>
>> >>> diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c
>> >>> index b65441d..23854b5 100644
>> >>> --- a/net/core/rtnetlink.c
>> >>> +++ b/net/core/rtnetlink.c
>> >>> @@ -1072,7 +1072,7 @@ static int rtnl_dump_ifinfo(struct sk_buff *skb, struct netlink_callback *cb)
>> >>> rcu_read_lock();
>> >>> cb->seq = net->dev_base_seq;
>> >>>
>> >>> - if (nlmsg_parse(cb->nlh, sizeof(struct rtgenmsg), tb, IFLA_MAX,
>> >>> + if (nlmsg_parse(cb->nlh, sizeof(struct ifinfomsg), tb, IFLA_MAX,
>> >>> ifla_policy) >= 0) {
>> >>>
>> >>> if (tb[IFLA_EXT_MASK])
>> >>> @@ -1922,7 +1922,7 @@ static u16 rtnl_calcit(struct sk_buff *skb, struct nlmsghdr *nlh)
>> >>> u32 ext_filter_mask = 0;
>> >>> u16 min_ifinfo_dump_size = 0;
>> >>>
>> >>> - if (nlmsg_parse(nlh, sizeof(struct rtgenmsg), tb, IFLA_MAX,
>> >>> + if (nlmsg_parse(nlh, sizeof(struct ifinfomsg), tb, IFLA_MAX,
>> >>> ifla_policy) >= 0) {
>> >>> if (tb[IFLA_EXT_MASK])
>> >>> ext_filter_mask = nla_get_u32(tb[IFLA_EXT_MASK]);
>> >> I thought that as well. I tried reverting it and the issue is still there.
>> >>
>> >> However, I do think this may be part of the issue since I added a printk
>> >> to dump nlmsg_attrlen before going into the nlmsg_parse and with
>> >> ifinfomsg the attrlen is -12, with rtgenmsg it is 0.
>> > I wonder if we are seeing two ways tools are making these calls, some are
>> > passing rtgenmsg and some are passing ifinfomsg. The latter, I am mostly
>> > convinced, is what we must see here from properly written applications.
>> >
>> > That would be really unfortunate, but seeing a nlmsg_attrlen() of -12 would
>> > seem to confirm that a rtgenmsg was used.
>> >
>> > I guess you're using iproute2?
>>
>> Yes. All I am doing is ip link show and previously this would display
>> VF info as well as PF. Now the VF info is not there.
>>
>> From what I can tell the problem call is rtnl_wilddump_req_filter since
>> it is only passing a rtgenmsg and asking us to dump the link with the
>> RTEXT_FILTER_VF filter mask.
>
> It looks like a bug in the initial code for filter. in this case, it calls:
> ip_list_flush_or_save
> rtnl_wilddump_req_filter
> which sends 'struct rtgenmsg'
>
> This was probably a mistake in the original flags addition, not sure if we can
> fix it now that ABI is set. Probably have to revert the kernel change.
But we know there are tools, just as widely deployed, passing in ifinfomsg.
That's what trigged inclusion of the patch above in the first place.
Let's just declare this an iproute2 bug, fix iproute2 to pass
ifinfomsg too, and keep an eye out for when other folks run into this
problem.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: getting VF link info seems to be broken in 3.9-rc8
2013-04-25 20:49 ` David Miller
@ 2013-04-25 21:10 ` Alexander Duyck
0 siblings, 0 replies; 20+ messages in thread
From: Alexander Duyck @ 2013-04-25 21:10 UTC (permalink / raw)
To: David Miller
Cc: stephen, mitch.a.williams, ogerlitz, gregory.v.rose, e1000-devel,
netdev, ronye, shemminger
On 04/25/2013 01:49 PM, David Miller wrote:
> From: Stephen Hemminger <stephen@networkplumber.org>
> Date: Thu, 25 Apr 2013 13:45:13 -0700
>
>> On Thu, 25 Apr 2013 13:36:06 -0700
>> Alexander Duyck <alexander.h.duyck@intel.com> wrote:
>>
>>> On 04/25/2013 01:25 PM, David Miller wrote:
>>>> From: Alexander Duyck <alexander.h.duyck@intel.com>
>>>> Date: Thu, 25 Apr 2013 13:20:24 -0700
>>>>
>>>>> On 04/25/2013 12:24 PM, David Miller wrote:
>>>>>> From: Alexander Duyck <alexander.h.duyck@intel.com>
>>>>>> Date: Thu, 25 Apr 2013 11:29:04 -0700
>>>>>>
>>>>>> diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c
>>>>>> index b65441d..23854b5 100644
>>>>>> --- a/net/core/rtnetlink.c
>>>>>> +++ b/net/core/rtnetlink.c
>>>>>> @@ -1072,7 +1072,7 @@ static int rtnl_dump_ifinfo(struct sk_buff *skb, struct netlink_callback *cb)
>>>>>> rcu_read_lock();
>>>>>> cb->seq = net->dev_base_seq;
>>>>>>
>>>>>> - if (nlmsg_parse(cb->nlh, sizeof(struct rtgenmsg), tb, IFLA_MAX,
>>>>>> + if (nlmsg_parse(cb->nlh, sizeof(struct ifinfomsg), tb, IFLA_MAX,
>>>>>> ifla_policy) >= 0) {
>>>>>>
>>>>>> if (tb[IFLA_EXT_MASK])
>>>>>> @@ -1922,7 +1922,7 @@ static u16 rtnl_calcit(struct sk_buff *skb, struct nlmsghdr *nlh)
>>>>>> u32 ext_filter_mask = 0;
>>>>>> u16 min_ifinfo_dump_size = 0;
>>>>>>
>>>>>> - if (nlmsg_parse(nlh, sizeof(struct rtgenmsg), tb, IFLA_MAX,
>>>>>> + if (nlmsg_parse(nlh, sizeof(struct ifinfomsg), tb, IFLA_MAX,
>>>>>> ifla_policy) >= 0) {
>>>>>> if (tb[IFLA_EXT_MASK])
>>>>>> ext_filter_mask = nla_get_u32(tb[IFLA_EXT_MASK]);
>>>>> I thought that as well. I tried reverting it and the issue is still there.
>>>>>
>>>>> However, I do think this may be part of the issue since I added a printk
>>>>> to dump nlmsg_attrlen before going into the nlmsg_parse and with
>>>>> ifinfomsg the attrlen is -12, with rtgenmsg it is 0.
>>>> I wonder if we are seeing two ways tools are making these calls, some are
>>>> passing rtgenmsg and some are passing ifinfomsg. The latter, I am mostly
>>>> convinced, is what we must see here from properly written applications.
>>>>
>>>> That would be really unfortunate, but seeing a nlmsg_attrlen() of -12 would
>>>> seem to confirm that a rtgenmsg was used.
>>>>
>>>> I guess you're using iproute2?
>>> Yes. All I am doing is ip link show and previously this would display
>>> VF info as well as PF. Now the VF info is not there.
>>>
>>> From what I can tell the problem call is rtnl_wilddump_req_filter since
>>> it is only passing a rtgenmsg and asking us to dump the link with the
>>> RTEXT_FILTER_VF filter mask.
>> It looks like a bug in the initial code for filter. in this case, it calls:
>> ip_list_flush_or_save
>> rtnl_wilddump_req_filter
>> which sends 'struct rtgenmsg'
>>
>> This was probably a mistake in the original flags addition, not sure if we can
>> fix it now that ABI is set. Probably have to revert the kernel change.
> But we know there are tools, just as widely deployed, passing in ifinfomsg.
> That's what trigged inclusion of the patch above in the first place.
>
> Let's just declare this an iproute2 bug, fix iproute2 to pass
> ifinfomsg too, and keep an eye out for when other folks run into this
> problem.
With the earlier patch reverted and the latest version iproute the
problem is resolved. I'm suspecting the other issue may have been a
64/32 bit alignment issue since it seems like that was addressed
recently. Odds are the message format for updated iproute will be
incompatible with legacy API anyway.
I will undo the revert and try tweaking the iproute2 call so that it
makes use of the ifinfomsg. If that works okay I can send a patch.
Thanks,
Alex
^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2013-04-25 21:10 UTC | newest]
Thread overview: 20+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-04-24 13:08 getting VF link info seems to be broken in 3.9-rc8 Or Gerlitz
2013-04-24 15:05 ` Williams, Mitch A
2013-04-25 18:29 ` Alexander Duyck
2013-04-25 19:24 ` David Miller
2013-04-25 20:20 ` Alexander Duyck
2013-04-25 20:25 ` David Miller
2013-04-25 20:36 ` Alexander Duyck
2013-04-25 20:45 ` Stephen Hemminger
2013-04-25 20:49 ` David Miller
2013-04-25 21:10 ` Alexander Duyck
2013-04-24 15:38 ` Greg Rose
2013-04-24 16:01 ` Greg Rose
2013-04-24 16:07 ` Or Gerlitz
2013-04-24 16:12 ` Li, Sibai
2013-04-24 16:08 ` Li, Sibai
2013-04-24 16:34 ` Or Gerlitz
2013-04-24 16:44 ` Li, Sibai
2013-04-24 16:45 ` Or Gerlitz
2013-04-24 16:48 ` Rose, Gregory V
2013-04-24 16:56 ` Li, Sibai
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).