bonding (IEEE 802.3ad) not working with qemu/virtio

bonding (IEEE 802.3ad) not working with qemu/virtio

[Bjørnar Ness]

As subject says, 802.3ad bonding is not working with virtio network model.

The only errors I see is:

No 802.3ad response from the link partner for any adapters in the bond.

Dumping the network traffic shows that no LACP packets are sent from the
host running with virtio driver, changing to for example e1000 solves
this problem
with no configuration changes.

Is this a known problem?

-- 
Bj(/)rnar

Re: bonding (IEEE 802.3ad) not working with qemu/virtio

[Jiri Pirko]

Mon, Jan 25, 2016 at 05:24:48PM CET, bjornar.ness@gmail.com wrote:
>As subject says, 802.3ad bonding is not working with virtio network model.
>
>The only errors I see is:
>
>No 802.3ad response from the link partner for any adapters in the bond.
>
>Dumping the network traffic shows that no LACP packets are sent from the
>host running with virtio driver, changing to for example e1000 solves
>this problem
>with no configuration changes.
>
>Is this a known problem?

I believe the problem is virtio_net for obvious reasons does not report
speed and duplex. Bonding 3ad mode makes that unconfortable :)

Use team ;)

Re: bonding (IEEE 802.3ad) not working with qemu/virtio

[Nikolay Aleksandrov]

On 01/28/2016 02:33 PM, Jiri Pirko wrote:
> Mon, Jan 25, 2016 at 05:24:48PM CET, bjornar.ness@gmail.com wrote:
>> As subject says, 802.3ad bonding is not working with virtio network model.
>>
>> The only errors I see is:
>>
>> No 802.3ad response from the link partner for any adapters in the bond.
>>
>> Dumping the network traffic shows that no LACP packets are sent from the
>> host running with virtio driver, changing to for example e1000 solves
>> this problem
>> with no configuration changes.
>>
>> Is this a known problem?
Can you show your bond's /proc/net/bonding/bond<X> ? And also in order to
better see what's going on I'd suggest enabling the pr_debug() calls in the 3ad
code:
echo 'file bond_3ad.c +p' > /sys/kernel/debug/dynamic_debug/control
(assuming you have debugfs mounted at /sys/kernel/debug)
Then you can follow the logs to see what's going on.
I can clearly see LACP packets sent over virtio net devices:
14:53:05.323490 52:54:00:51:25:3c > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110

> 
> I believe the problem is virtio_net for obvious reasons does not report
> speed and duplex. Bonding 3ad mode makes that unconfortable :)
root@dev:~# ethtool -i eth1
driver: virtio_net
root@dev:~# ethtool eth1
Settings for eth1:
	Supported ports: [ ]
	Supported link modes:   Not reported
	Supported pause frame use: No
	Supports auto-negotiation: No
	Advertised link modes:  Not reported
	Advertised pause frame use: No
	Advertised auto-negotiation: No
	*Speed: 10Mb/s*
	*Duplex: Full*
	Port: Twisted Pair
	PHYAD: 0
	Transceiver: internal
	Auto-negotiation: off
	MDI-X: Unknown
	Link detected: yes

The bonding catches that correctly,
[54569.138572] bond0: Adding slave eth1
[54569.139686] bond0: Port 1 Received status full duplex update from adapter
[54569.139690] bond0: Port 1 Received link speed 2 update from adapter

The debug messages are from enabled 3ad mode pr_debugs().
Added 2 virtio_net adapters and they successfully went in a single LAG
when it was enabled on the other side.
root@dev:~# cat /proc/net/bonding/bond0 
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

802.3ad info
LACP rate: slow
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
System MAC address: 52:54:00:51:25:3c
Active Aggregator Info:
	Aggregator ID: 1
	Number of ports: 2
	Actor Key: 5
	Partner Key: 0
	Partner Mac Address: 52:54:00:2f:30:f7

Slave Interface: eth1
MII Status: up
Speed: 10 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 52:54:00:51:25:3c
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: monitoring
Partner Churn State: monitoring
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
    system priority: 65535
    system mac address: 52:54:00:51:25:3c
    port key: 5
    port priority: 255
    port number: 1
    port state: 61
details partner lacp pdu:
    system priority: 65535
    system mac address: 52:54:00:2f:30:f7
    oper key: 0
    port priority: 255
    port number: 1
    port state: 73

Slave Interface: eth2
MII Status: up
Speed: 10 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 52:54:00:bf:57:16
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: monitoring
Partner Churn State: monitoring
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
    system priority: 65535
    system mac address: 52:54:00:51:25:3c
    port key: 5
    port priority: 255
    port number: 2
    port state: 61
details partner lacp pdu:
    system priority: 65535
    system mac address: 52:54:00:2f:30:f7
    oper key: 0
    port priority: 255
    port number: 1
    port state: 73

> 
> Use team ;)

Re: bonding (IEEE 802.3ad) not working with qemu/virtio

[Nikolay Aleksandrov]

On 01/28/2016 02:53 PM, Nikolay Aleksandrov wrote:
> On 01/28/2016 02:33 PM, Jiri Pirko wrote:
>> Mon, Jan 25, 2016 at 05:24:48PM CET, bjornar.ness@gmail.com wrote:
>>> As subject says, 802.3ad bonding is not working with virtio network model.
>>>
>>> The only errors I see is:
>>>
>>> No 802.3ad response from the link partner for any adapters in the bond.
>>>
>>> Dumping the network traffic shows that no LACP packets are sent from the
>>> host running with virtio driver, changing to for example e1000 solves
>>> this problem
>>> with no configuration changes.
>>>
>>> Is this a known problem?
> Can you show your bond's /proc/net/bonding/bond<X> ? And also in order to
> better see what's going on I'd suggest enabling the pr_debug() calls in the 3ad
> code:
> echo 'file bond_3ad.c +p' > /sys/kernel/debug/dynamic_debug/control
> (assuming you have debugfs mounted at /sys/kernel/debug)
> Then you can follow the logs to see what's going on.
> I can clearly see LACP packets sent over virtio net devices:
> 14:53:05.323490 52:54:00:51:25:3c > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
Nevermind my message, Jiri was right. My setup wasn't correct.
I have a virtio_net get_settings hack that enables me to test 802.3ad over virtio_net
locally.
Sorry about the noise.

> 
>>
>> I believe the problem is virtio_net for obvious reasons does not report
>> speed and duplex. Bonding 3ad mode makes that unconfortable :)
[snip]
Right, of course. My setup has some patches that add get_settings to virtio_net
to make it play nice for testing 803.2ad.

>>
>> Use team ;)
> 
:-)

Re: bonding (IEEE 802.3ad) not working with qemu/virtio

[Nikolay Aleksandrov]

On 01/25/2016 05:24 PM, Bjørnar Ness wrote:
> As subject says, 802.3ad bonding is not working with virtio network model.
> 
> The only errors I see is:
> 
> No 802.3ad response from the link partner for any adapters in the bond.
> 
> Dumping the network traffic shows that no LACP packets are sent from the
> host running with virtio driver, changing to for example e1000 solves
> this problem
> with no configuration changes.
> 
> Is this a known problem?
> 
[Including bonding maintainers for comments]

Hi,
Here's a workaround patch for virtio_net devices that "cheats" the
duplex test (which is the actual problem). I've tested this locally
and it works for me.
I'd let the others comment on the implementation, there're other signs
that can be used to distinguish a virtio_net device so I'm open to suggestions.
Also feedback if this is at all acceptable would be appreciated.

Could you give it a try ?

diff --git a/drivers/net/bonding/bond_3ad.c b/drivers/net/bonding/bond_3ad.c
index 4cbb8b27a891..0578a95e3ade 100644
--- a/drivers/net/bonding/bond_3ad.c
+++ b/drivers/net/bonding/bond_3ad.c
@@ -334,6 +334,7 @@ static u16 __get_link_speed(struct port *port)
 static u8 __get_duplex(struct port *port)
 {
 	struct slave *slave = port->slave;
+	const struct net_device *slave_dev = slave->dev;
 	u8 retval = 0x0;
 
 	/* handling a special case: when the configuration starts with
@@ -354,6 +355,15 @@ static u8 __get_duplex(struct port *port)
 			break;
 		}
 	}
+	if (!retval && slave_dev->dev.parent &&
+	    !strcmp(dev_driver_string(slave_dev->dev.parent),
+		    "virtio_net") &&
+	    netif_carrier_ok(slave_dev)) {
+		netdev_dbg(slave->bond->dev, "port %d: activating virtio_net workaround\n",
+			   port->actor_port_number);
+		retval = 1;
+	}
+
 	return retval;
 }
 

Re: bonding (IEEE 802.3ad) not working with qemu/virtio

[Jay Vosburgh]

Nikolay Aleksandrov <nikolay@cumulusnetworks.com> wrote:

>On 01/25/2016 05:24 PM, Bjørnar Ness wrote:
>> As subject says, 802.3ad bonding is not working with virtio network model.
>> 
>> The only errors I see is:
>> 
>> No 802.3ad response from the link partner for any adapters in the bond.
>> 
>> Dumping the network traffic shows that no LACP packets are sent from the
>> host running with virtio driver, changing to for example e1000 solves
>> this problem
>> with no configuration changes.
>> 
>> Is this a known problem?
>> 
>[Including bonding maintainers for comments]
>
>Hi,
>Here's a workaround patch for virtio_net devices that "cheats" the
>duplex test (which is the actual problem). I've tested this locally
>and it works for me.
>I'd let the others comment on the implementation, there're other signs
>that can be used to distinguish a virtio_net device so I'm open to suggestions.
>Also feedback if this is at all acceptable would be appreciated.

	Should virtio instead provide an arbitrary speed and full duplex
to ethtool, as veth does?

	Creating a magic whitelist of devices deep inside the 802.3ad
implementation seems less desirable.

	-J
	
---
	-Jay Vosburgh, jay.vosburgh@canonical.com

Re: bonding (IEEE 802.3ad) not working with qemu/virtio

[Nikolay Aleksandrov]

On 01/29/2016 10:45 PM, Jay Vosburgh wrote:
> Nikolay Aleksandrov <nikolay@cumulusnetworks.com> wrote:
> 
>> On 01/25/2016 05:24 PM, Bjørnar Ness wrote:
>>> As subject says, 802.3ad bonding is not working with virtio network model.
>>>
>>> The only errors I see is:
>>>
>>> No 802.3ad response from the link partner for any adapters in the bond.
>>>
>>> Dumping the network traffic shows that no LACP packets are sent from the
>>> host running with virtio driver, changing to for example e1000 solves
>>> this problem
>>> with no configuration changes.
>>>
>>> Is this a known problem?
>>>
>> [Including bonding maintainers for comments]
>>
>> Hi,
>> Here's a workaround patch for virtio_net devices that "cheats" the
>> duplex test (which is the actual problem). I've tested this locally
>> and it works for me.
>> I'd let the others comment on the implementation, there're other signs
>> that can be used to distinguish a virtio_net device so I'm open to suggestions.
>> Also feedback if this is at all acceptable would be appreciated.
> 
> 	Should virtio instead provide an arbitrary speed and full duplex
> to ethtool, as veth does?
> 
> 	Creating a magic whitelist of devices deep inside the 802.3ad
> implementation seems less desirable.
> 
TBH, I absolutely agree. In fact here's what we've been doing:
add set_settings which allows the user to set any speed/duplex
and get_settings of course to retrieve that. This is also useful
for testing other stuff that requires speed and duplex, not only
for the bonding case.

I'll add the virtio_net maintainers to the discussion, see if it's
okay with everyone and I'll move to send patches once net-next opens up.

Thanks!


> 	-J
> 	
> ---
> 	-Jay Vosburgh, jay.vosburgh@canonical.com
> 

Re: bonding (IEEE 802.3ad) not working with qemu/virtio

[David Miller]

From: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
Date: Fri, 29 Jan 2016 22:48:26 +0100

> On 01/29/2016 10:45 PM, Jay Vosburgh wrote:
>> Nikolay Aleksandrov <nikolay@cumulusnetworks.com> wrote:
>> 
>>> On 01/25/2016 05:24 PM, Bjørnar Ness wrote:
>>>> As subject says, 802.3ad bonding is not working with virtio network model.
>>>>
>>>> The only errors I see is:
>>>>
>>>> No 802.3ad response from the link partner for any adapters in the bond.
>>>>
>>>> Dumping the network traffic shows that no LACP packets are sent from the
>>>> host running with virtio driver, changing to for example e1000 solves
>>>> this problem
>>>> with no configuration changes.
>>>>
>>>> Is this a known problem?
>>>>
>>> [Including bonding maintainers for comments]
>>>
>>> Hi,
>>> Here's a workaround patch for virtio_net devices that "cheats" the
>>> duplex test (which is the actual problem). I've tested this locally
>>> and it works for me.
>>> I'd let the others comment on the implementation, there're other signs
>>> that can be used to distinguish a virtio_net device so I'm open to suggestions.
>>> Also feedback if this is at all acceptable would be appreciated.
>> 
>> 	Should virtio instead provide an arbitrary speed and full duplex
>> to ethtool, as veth does?
>> 
>> 	Creating a magic whitelist of devices deep inside the 802.3ad
>> implementation seems less desirable.
>> 
> TBH, I absolutely agree. In fact here's what we've been doing:
> add set_settings which allows the user to set any speed/duplex
> and get_settings of course to retrieve that. This is also useful
> for testing other stuff that requires speed and duplex, not only
> for the bonding case.

I also agree.  Having a whitelist is just rediculous.

There should be a default speed/duplex setting for such devices as well.
We can pick one that will be use universally for these kinds of devices.

Re: bonding (IEEE 802.3ad) not working with qemu/virtio

[Jiri Pirko]

Sat, Jan 30, 2016 at 07:59:24AM CET, davem@davemloft.net wrote:
>From: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
>Date: Fri, 29 Jan 2016 22:48:26 +0100
>
>> On 01/29/2016 10:45 PM, Jay Vosburgh wrote:
>>> Nikolay Aleksandrov <nikolay@cumulusnetworks.com> wrote:
>>> 
>>>> On 01/25/2016 05:24 PM, Bjørnar Ness wrote:
>>>>> As subject says, 802.3ad bonding is not working with virtio network model.
>>>>>
>>>>> The only errors I see is:
>>>>>
>>>>> No 802.3ad response from the link partner for any adapters in the bond.
>>>>>
>>>>> Dumping the network traffic shows that no LACP packets are sent from the
>>>>> host running with virtio driver, changing to for example e1000 solves
>>>>> this problem
>>>>> with no configuration changes.
>>>>>
>>>>> Is this a known problem?
>>>>>
>>>> [Including bonding maintainers for comments]
>>>>
>>>> Hi,
>>>> Here's a workaround patch for virtio_net devices that "cheats" the
>>>> duplex test (which is the actual problem). I've tested this locally
>>>> and it works for me.
>>>> I'd let the others comment on the implementation, there're other signs
>>>> that can be used to distinguish a virtio_net device so I'm open to suggestions.
>>>> Also feedback if this is at all acceptable would be appreciated.
>>> 
>>> 	Should virtio instead provide an arbitrary speed and full duplex
>>> to ethtool, as veth does?
>>> 
>>> 	Creating a magic whitelist of devices deep inside the 802.3ad
>>> implementation seems less desirable.
>>> 
>> TBH, I absolutely agree. In fact here's what we've been doing:
>> add set_settings which allows the user to set any speed/duplex
>> and get_settings of course to retrieve that. This is also useful
>> for testing other stuff that requires speed and duplex, not only
>> for the bonding case.
>
>I also agree.  Having a whitelist is just rediculous.
>
>There should be a default speed/duplex setting for such devices as well.
>We can pick one that will be use universally for these kinds of devices.

Exposing made up speed for veth and virtio_net sounds odd to me. User
see 10000Mb/s but it makes no sense at all. It is just confusing.

I believe this is bonding bug and should be fixed in there. Team works
fine with virtio_net device and lacp runner.

Re: bonding (IEEE 802.3ad) not working with qemu/virtio

[Jiri Pirko]

Sat, Jan 30, 2016 at 07:59:24AM CET, davem@davemloft.net wrote:
>From: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
>Date: Fri, 29 Jan 2016 22:48:26 +0100
>
>> On 01/29/2016 10:45 PM, Jay Vosburgh wrote:
>>> Nikolay Aleksandrov <nikolay@cumulusnetworks.com> wrote:
>>> 
>>>> On 01/25/2016 05:24 PM, Bjørnar Ness wrote:
>>>>> As subject says, 802.3ad bonding is not working with virtio network model.
>>>>>
>>>>> The only errors I see is:
>>>>>
>>>>> No 802.3ad response from the link partner for any adapters in the bond.
>>>>>
>>>>> Dumping the network traffic shows that no LACP packets are sent from the
>>>>> host running with virtio driver, changing to for example e1000 solves
>>>>> this problem
>>>>> with no configuration changes.
>>>>>
>>>>> Is this a known problem?
>>>>>
>>>> [Including bonding maintainers for comments]
>>>>
>>>> Hi,
>>>> Here's a workaround patch for virtio_net devices that "cheats" the
>>>> duplex test (which is the actual problem). I've tested this locally
>>>> and it works for me.
>>>> I'd let the others comment on the implementation, there're other signs
>>>> that can be used to distinguish a virtio_net device so I'm open to suggestions.
>>>> Also feedback if this is at all acceptable would be appreciated.
>>> 
>>> 	Should virtio instead provide an arbitrary speed and full duplex
>>> to ethtool, as veth does?
>>> 
>>> 	Creating a magic whitelist of devices deep inside the 802.3ad
>>> implementation seems less desirable.
>>> 
>> TBH, I absolutely agree. In fact here's what we've been doing:
>> add set_settings which allows the user to set any speed/duplex
>> and get_settings of course to retrieve that. This is also useful
>> for testing other stuff that requires speed and duplex, not only
>> for the bonding case.
>
>I also agree.  Having a whitelist is just rediculous.
>
>There should be a default speed/duplex setting for such devices as well.
>We can pick one that will be use universally for these kinds of devices.

Exposing made up speed for veth and virtio_net sounds odd to me. User
see 10000Mb/s but it makes no sense at all. It is just confusing.

I believe this is bonding bug and should be fixed in there. Team works
fine with virtio_net device and lacp runner.

Re: bonding (IEEE 802.3ad) not working with qemu/virtio

[Nikolay Aleksandrov]

On 01/30/2016 07:59 AM, David Miller wrote:
> From: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
> Date: Fri, 29 Jan 2016 22:48:26 +0100
> 
>> On 01/29/2016 10:45 PM, Jay Vosburgh wrote:
>>> Nikolay Aleksandrov <nikolay@cumulusnetworks.com> wrote:
>>>
>>>> On 01/25/2016 05:24 PM, Bjørnar Ness wrote:
>>>>> As subject says, 802.3ad bonding is not working with virtio network model.
>>>>>
>>>>> The only errors I see is:
>>>>>
>>>>> No 802.3ad response from the link partner for any adapters in the bond.
>>>>>
>>>>> Dumping the network traffic shows that no LACP packets are sent from the
>>>>> host running with virtio driver, changing to for example e1000 solves
>>>>> this problem
>>>>> with no configuration changes.
>>>>>
>>>>> Is this a known problem?
>>>>>
>>>> [Including bonding maintainers for comments]
>>>>
>>>> Hi,
>>>> Here's a workaround patch for virtio_net devices that "cheats" the
>>>> duplex test (which is the actual problem). I've tested this locally
>>>> and it works for me.
>>>> I'd let the others comment on the implementation, there're other signs
>>>> that can be used to distinguish a virtio_net device so I'm open to suggestions.
>>>> Also feedback if this is at all acceptable would be appreciated.
>>>
>>> 	Should virtio instead provide an arbitrary speed and full duplex
>>> to ethtool, as veth does?
>>>
>>> 	Creating a magic whitelist of devices deep inside the 802.3ad
>>> implementation seems less desirable.
>>>
>> TBH, I absolutely agree. In fact here's what we've been doing:
>> add set_settings which allows the user to set any speed/duplex
>> and get_settings of course to retrieve that. This is also useful
>> for testing other stuff that requires speed and duplex, not only
>> for the bonding case.
> 
> I also agree.  Having a whitelist is just rediculous.
> 
> There should be a default speed/duplex setting for such devices as well.
> We can pick one that will be use universally for these kinds of devices.
> 
Yes, that's the other thing - the default setting, from a brief grepping
I see that veth uses 10Gbps, tun uses 10Mbps and batman-adv uses 10Mbps.

If we add a default get_settings that can be used by virtual devices
in ethtool that returns 10Gbps with the settings set like veth does
sounds good to me. What do you think ?

In fact they all set the same settings (apart from speed) so we can
consolidate them in a single default setting init function and for the
ones using different speed do something like:
__ethtool_init_settings(); <- sets everything like veth would with 10Mbps speed
__ethtool_cmd_speed_set(cmd, SPEED_10000);
And we're done.

Re: bonding (IEEE 802.3ad) not working with qemu/virtio

[Nikolay Aleksandrov]

On 01/30/2016 07:59 AM, David Miller wrote:
> From: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
> Date: Fri, 29 Jan 2016 22:48:26 +0100
> 
>> On 01/29/2016 10:45 PM, Jay Vosburgh wrote:
>>> Nikolay Aleksandrov <nikolay@cumulusnetworks.com> wrote:
>>>
>>>> On 01/25/2016 05:24 PM, Bjørnar Ness wrote:
>>>>> As subject says, 802.3ad bonding is not working with virtio network model.
>>>>>
>>>>> The only errors I see is:
>>>>>
>>>>> No 802.3ad response from the link partner for any adapters in the bond.
>>>>>
>>>>> Dumping the network traffic shows that no LACP packets are sent from the
>>>>> host running with virtio driver, changing to for example e1000 solves
>>>>> this problem
>>>>> with no configuration changes.
>>>>>
>>>>> Is this a known problem?
>>>>>
>>>> [Including bonding maintainers for comments]
>>>>
>>>> Hi,
>>>> Here's a workaround patch for virtio_net devices that "cheats" the
>>>> duplex test (which is the actual problem). I've tested this locally
>>>> and it works for me.
>>>> I'd let the others comment on the implementation, there're other signs
>>>> that can be used to distinguish a virtio_net device so I'm open to suggestions.
>>>> Also feedback if this is at all acceptable would be appreciated.
>>>
>>> 	Should virtio instead provide an arbitrary speed and full duplex
>>> to ethtool, as veth does?
>>>
>>> 	Creating a magic whitelist of devices deep inside the 802.3ad
>>> implementation seems less desirable.
>>>
>> TBH, I absolutely agree. In fact here's what we've been doing:
>> add set_settings which allows the user to set any speed/duplex
>> and get_settings of course to retrieve that. This is also useful
>> for testing other stuff that requires speed and duplex, not only
>> for the bonding case.
> 
> I also agree.  Having a whitelist is just rediculous.
> 
> There should be a default speed/duplex setting for such devices as well.
> We can pick one that will be use universally for these kinds of devices.
> 
Yes, that's the other thing - the default setting, from a brief grepping
I see that veth uses 10Gbps, tun uses 10Mbps and batman-adv uses 10Mbps.

If we add a default get_settings that can be used by virtual devices
in ethtool that returns 10Gbps with the settings set like veth does
sounds good to me. What do you think ?

In fact they all set the same settings (apart from speed) so we can
consolidate them in a single default setting init function and for the
ones using different speed do something like:
__ethtool_init_settings(); <- sets everything like veth would with 10Mbps speed
__ethtool_cmd_speed_set(cmd, SPEED_10000);
And we're done.

Re: bonding (IEEE 802.3ad) not working with qemu/virtio

[Michael S. Tsirkin]

On Sat, Jan 30, 2016 at 12:41:16PM +0100, Nikolay Aleksandrov wrote:
> On 01/30/2016 07:59 AM, David Miller wrote:
> > From: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
> > Date: Fri, 29 Jan 2016 22:48:26 +0100
> > 
> >> On 01/29/2016 10:45 PM, Jay Vosburgh wrote:
> >>> Nikolay Aleksandrov <nikolay@cumulusnetworks.com> wrote:
> >>>
> >>>> On 01/25/2016 05:24 PM, Bjørnar Ness wrote:
> >>>>> As subject says, 802.3ad bonding is not working with virtio network model.
> >>>>>
> >>>>> The only errors I see is:
> >>>>>
> >>>>> No 802.3ad response from the link partner for any adapters in the bond.
> >>>>>
> >>>>> Dumping the network traffic shows that no LACP packets are sent from the
> >>>>> host running with virtio driver, changing to for example e1000 solves
> >>>>> this problem
> >>>>> with no configuration changes.
> >>>>>
> >>>>> Is this a known problem?
> >>>>>
> >>>> [Including bonding maintainers for comments]
> >>>>
> >>>> Hi,
> >>>> Here's a workaround patch for virtio_net devices that "cheats" the
> >>>> duplex test (which is the actual problem). I've tested this locally
> >>>> and it works for me.
> >>>> I'd let the others comment on the implementation, there're other signs
> >>>> that can be used to distinguish a virtio_net device so I'm open to suggestions.
> >>>> Also feedback if this is at all acceptable would be appreciated.
> >>>
> >>> 	Should virtio instead provide an arbitrary speed and full duplex
> >>> to ethtool, as veth does?
> >>>
> >>> 	Creating a magic whitelist of devices deep inside the 802.3ad
> >>> implementation seems less desirable.
> >>>
> >> TBH, I absolutely agree. In fact here's what we've been doing:
> >> add set_settings which allows the user to set any speed/duplex
> >> and get_settings of course to retrieve that. This is also useful
> >> for testing other stuff that requires speed and duplex, not only
> >> for the bonding case.
> > 
> > I also agree.  Having a whitelist is just rediculous.
> > 
> > There should be a default speed/duplex setting for such devices as well.
> > We can pick one that will be use universally for these kinds of devices.
> > 
> Yes, that's the other thing - the default setting, from a brief grepping
> I see that veth uses 10Gbps, tun uses 10Mbps and batman-adv uses 10Mbps.
> 
> If we add a default get_settings that can be used by virtual devices
> in ethtool that returns 10Gbps with the settings set like veth does
> sounds good to me. What do you think ?
> 
> In fact they all set the same settings (apart from speed) so we can
> consolidate them in a single default setting init function and for the
> ones using different speed do something like:
> __ethtool_init_settings(); <- sets everything like veth would with 10Mbps speed
> __ethtool_cmd_speed_set(cmd, SPEED_10000);
> And we're done.
> 

I think leaving speed alone would be preferable if possible.
Faking speed might cause strange effects e.g. bonding prefering
virtio over a real device unexpectedly.
Also, wouldn't it break same link speed requirements?
Same applies to duplex really.

We could easily extend virtio with ability to pass
speed/duplex info from host, but that's not always
known to host and there are existing hypervisors,
so we are still left with the question of what to do
when this info is not available.


Is virtio somehow special that we want LACP for it
if duplex state and speed are unknown, but
don't want it for other devices?
If no, let's allow it for all such devices?

-->
bond_3ad: allow LACP when duplex is unknown

Devices would sometimes set speed and duplex as unknown.

It seem reasonable to allow LACP in this case.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

--

diff --git a/drivers/net/bonding/bond_3ad.c b/drivers/net/bonding/bond_3ad.c
index 4cbb8b2..92639ef 100644
--- a/drivers/net/bonding/bond_3ad.c
+++ b/drivers/net/bonding/bond_3ad.c
@@ -342,12 +342,12 @@ static u8 __get_duplex(struct port *port)
 	if (slave->link == BOND_LINK_UP) {
 		switch (slave->duplex) {
 		case DUPLEX_FULL:
+		default:
 			retval = 0x1;
 			netdev_dbg(slave->bond->dev, "Port %d Received status full duplex update from adapter\n",
 				   port->actor_port_number);
 			break;
 		case DUPLEX_HALF:
-		default:
 			retval = 0x0;
 			netdev_dbg(slave->bond->dev, "Port %d Received status NOT full duplex update from adapter\n",
 				   port->actor_port_number);

-- 
MST

Re: bonding (IEEE 802.3ad) not working with qemu/virtio

[Michael S. Tsirkin]

On Sat, Jan 30, 2016 at 12:41:16PM +0100, Nikolay Aleksandrov wrote:
> On 01/30/2016 07:59 AM, David Miller wrote:
> > From: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
> > Date: Fri, 29 Jan 2016 22:48:26 +0100
> > 
> >> On 01/29/2016 10:45 PM, Jay Vosburgh wrote:
> >>> Nikolay Aleksandrov <nikolay@cumulusnetworks.com> wrote:
> >>>
> >>>> On 01/25/2016 05:24 PM, Bjørnar Ness wrote:
> >>>>> As subject says, 802.3ad bonding is not working with virtio network model.
> >>>>>
> >>>>> The only errors I see is:
> >>>>>
> >>>>> No 802.3ad response from the link partner for any adapters in the bond.
> >>>>>
> >>>>> Dumping the network traffic shows that no LACP packets are sent from the
> >>>>> host running with virtio driver, changing to for example e1000 solves
> >>>>> this problem
> >>>>> with no configuration changes.
> >>>>>
> >>>>> Is this a known problem?
> >>>>>
> >>>> [Including bonding maintainers for comments]
> >>>>
> >>>> Hi,
> >>>> Here's a workaround patch for virtio_net devices that "cheats" the
> >>>> duplex test (which is the actual problem). I've tested this locally
> >>>> and it works for me.
> >>>> I'd let the others comment on the implementation, there're other signs
> >>>> that can be used to distinguish a virtio_net device so I'm open to suggestions.
> >>>> Also feedback if this is at all acceptable would be appreciated.
> >>>
> >>> 	Should virtio instead provide an arbitrary speed and full duplex
> >>> to ethtool, as veth does?
> >>>
> >>> 	Creating a magic whitelist of devices deep inside the 802.3ad
> >>> implementation seems less desirable.
> >>>
> >> TBH, I absolutely agree. In fact here's what we've been doing:
> >> add set_settings which allows the user to set any speed/duplex
> >> and get_settings of course to retrieve that. This is also useful
> >> for testing other stuff that requires speed and duplex, not only
> >> for the bonding case.
> > 
> > I also agree.  Having a whitelist is just rediculous.
> > 
> > There should be a default speed/duplex setting for such devices as well.
> > We can pick one that will be use universally for these kinds of devices.
> > 
> Yes, that's the other thing - the default setting, from a brief grepping
> I see that veth uses 10Gbps, tun uses 10Mbps and batman-adv uses 10Mbps.
> 
> If we add a default get_settings that can be used by virtual devices
> in ethtool that returns 10Gbps with the settings set like veth does
> sounds good to me. What do you think ?
> 
> In fact they all set the same settings (apart from speed) so we can
> consolidate them in a single default setting init function and for the
> ones using different speed do something like:
> __ethtool_init_settings(); <- sets everything like veth would with 10Mbps speed
> __ethtool_cmd_speed_set(cmd, SPEED_10000);
> And we're done.
> 

I think leaving speed alone would be preferable if possible.
Faking speed might cause strange effects e.g. bonding prefering
virtio over a real device unexpectedly.
Also, wouldn't it break same link speed requirements?
Same applies to duplex really.

We could easily extend virtio with ability to pass
speed/duplex info from host, but that's not always
known to host and there are existing hypervisors,
so we are still left with the question of what to do
when this info is not available.


Is virtio somehow special that we want LACP for it
if duplex state and speed are unknown, but
don't want it for other devices?
If no, let's allow it for all such devices?

-->
bond_3ad: allow LACP when duplex is unknown

Devices would sometimes set speed and duplex as unknown.

It seem reasonable to allow LACP in this case.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

Re: bonding (IEEE 802.3ad) not working with qemu/virtio

[Rick Jones]

On 01/29/2016 10:59 PM, David Miller wrote:
> There should be a default speed/duplex setting for such devices as well.
> We can pick one that will be use universally for these kinds of devices.

There is at least one monitoring tool - collectl - which gets a trifle 
upset when the actual speed through an interface is significantly 
greater than the reported link speed.  I have to wonder how unique it is 
in that regard.

Doesn't mean there can't be a default, but does suggest it should be 
rather high.

rick jones

Re: bonding (IEEE 802.3ad) not working with qemu/virtio

[Michael S. Tsirkin]

On Fri, Jan 29, 2016 at 10:48:26PM +0100, Nikolay Aleksandrov wrote:
> On 01/29/2016 10:45 PM, Jay Vosburgh wrote:
> > Nikolay Aleksandrov <nikolay@cumulusnetworks.com> wrote:
> > 
> >> On 01/25/2016 05:24 PM, Bjørnar Ness wrote:
> >>> As subject says, 802.3ad bonding is not working with virtio network model.
> >>>
> >>> The only errors I see is:
> >>>
> >>> No 802.3ad response from the link partner for any adapters in the bond.
> >>>
> >>> Dumping the network traffic shows that no LACP packets are sent from the
> >>> host running with virtio driver, changing to for example e1000 solves
> >>> this problem
> >>> with no configuration changes.
> >>>
> >>> Is this a known problem?
> >>>
> >> [Including bonding maintainers for comments]
> >>
> >> Hi,
> >> Here's a workaround patch for virtio_net devices that "cheats" the
> >> duplex test (which is the actual problem). I've tested this locally
> >> and it works for me.
> >> I'd let the others comment on the implementation, there're other signs
> >> that can be used to distinguish a virtio_net device so I'm open to suggestions.
> >> Also feedback if this is at all acceptable would be appreciated.
> > 
> > 	Should virtio instead provide an arbitrary speed and full duplex
> > to ethtool, as veth does?
> > 
> > 	Creating a magic whitelist of devices deep inside the 802.3ad
> > implementation seems less desirable.
> > 
> TBH, I absolutely agree. In fact here's what we've been doing:
> add set_settings which allows the user to set any speed/duplex
> and get_settings of course to retrieve that. This is also useful
> for testing other stuff that requires speed and duplex, not only
> for the bonding case.

This looks like a very reasonable thing to do:
user might have knowledge of the actual speed through
some side-channel.

We might also propagate it to hypervisor in the future.

And this sound useful even if bonding is changed
to allow DUPLEX_UNKNOWN.

So please post this patch.


> I'll add the virtio_net maintainers to the discussion, see if it's
> okay with everyone and I'll move to send patches once net-next opens up.
> 
> Thanks!
> 
> 
> > 	-J
> > 	
> > ---
> > 	-Jay Vosburgh, jay.vosburgh@canonical.com
> > 

Re: bonding (IEEE 802.3ad) not working with qemu/virtio

[Nikolay Aleksandrov]

On 01/29/2016 10:45 PM, Jay Vosburgh wrote:
> Nikolay Aleksandrov <nikolay@cumulusnetworks.com> wrote:
> 
>> On 01/25/2016 05:24 PM, Bjørnar Ness wrote:
>>> As subject says, 802.3ad bonding is not working with virtio network model.
>>>
>>> The only errors I see is:
>>>
>>> No 802.3ad response from the link partner for any adapters in the bond.
>>>
>>> Dumping the network traffic shows that no LACP packets are sent from the
>>> host running with virtio driver, changing to for example e1000 solves
>>> this problem
>>> with no configuration changes.
>>>
>>> Is this a known problem?
>>>
>> [Including bonding maintainers for comments]
>>
>> Hi,
>> Here's a workaround patch for virtio_net devices that "cheats" the
>> duplex test (which is the actual problem). I've tested this locally
>> and it works for me.
>> I'd let the others comment on the implementation, there're other signs
>> that can be used to distinguish a virtio_net device so I'm open to suggestions.
>> Also feedback if this is at all acceptable would be appreciated.
> 
> 	Should virtio instead provide an arbitrary speed and full duplex
> to ethtool, as veth does?
> 
> 	Creating a magic whitelist of devices deep inside the 802.3ad
> implementation seems less desirable.
> 
TBH, I absolutely agree. In fact here's what we've been doing:
add set_settings which allows the user to set any speed/duplex
and get_settings of course to retrieve that. This is also useful
for testing other stuff that requires speed and duplex, not only
for the bonding case.

I'll add the virtio_net maintainers to the discussion, see if it's
okay with everyone and I'll move to send patches once net-next opens up.

Thanks!


> 	-J
> 	
> ---
> 	-Jay Vosburgh, jay.vosburgh@canonical.com
> 

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: bonding (IEEE 802.3ad) not working with qemu/virtio

[Michael S. Tsirkin]

On Fri, Jan 29, 2016 at 01:45:00PM -0800, Jay Vosburgh wrote:
> Nikolay Aleksandrov <nikolay@cumulusnetworks.com> wrote:
> 
> >On 01/25/2016 05:24 PM, Bjørnar Ness wrote:
> >> As subject says, 802.3ad bonding is not working with virtio network model.
> >> 
> >> The only errors I see is:
> >> 
> >> No 802.3ad response from the link partner for any adapters in the bond.
> >> 
> >> Dumping the network traffic shows that no LACP packets are sent from the
> >> host running with virtio driver, changing to for example e1000 solves
> >> this problem
> >> with no configuration changes.
> >> 
> >> Is this a known problem?
> >> 
> >[Including bonding maintainers for comments]
> >
> >Hi,
> >Here's a workaround patch for virtio_net devices that "cheats" the
> >duplex test (which is the actual problem). I've tested this locally
> >and it works for me.
> >I'd let the others comment on the implementation, there're other signs
> >that can be used to distinguish a virtio_net device so I'm open to suggestions.
> >Also feedback if this is at all acceptable would be appreciated.
> 
> 	Should virtio instead provide an arbitrary speed and full duplex
> to ethtool, as veth does?
> 
> 	Creating a magic whitelist of devices deep inside the 802.3ad
> implementation seems less desirable.
> 
> 	-J

Absolutely but why not tream DUPLEX_UNKNOWN as DUPLEX_FULL
and allow LACP?

> ---
> 	-Jay Vosburgh, jay.vosburgh@canonical.com