BUG: Bisected Gianfar in bridge not forwarding packets (was: 3.0-rc1 Bridge not forwarding unicast packages)

BUG: Bisected Gianfar in bridge not forwarding packets (was: 3.0-rc1 Bridge not forwarding unicast packages)

[Michael Guntsche]

Good evening/morning,

> I just upgraded my router/bridge combo to 3.1-rc1 from 3.0 for
> testing. On a first look everything seemed to work fine, but when I
> tried to connect via openvpn to my internal network (tap0 being bridged
> with the internal network) I noticed that I was not able to access the
> server on my internal network. I could access the bridge (which is
> acting as the openvpn server as well) just fine though.
> To debug this I ran tcpdump on the openvpn client and started a ping to the
> internal network. I could see the ARP requests being answered.....
<snip>

After much digging and testing I found the commit that causes the
problem for me here.

87c288c6e gianfar: do vlan cleanup

Before this commit my bridge works fine. I simplified the test case
and just tried to access the server from a local wireless device with
gets also bridged via
a gianfar nic to the wired lan. Before commit 87c288c6e I can ping the
device from the internal network afterwards it stops.
For testing purposes I reverted this commit (plus some others to make
the code compile) and it started working again.
The hardware is a routerboard with three onboard nics two of em gianfar.
The gianfar nics do not support any kind of offloading,

Offload parameters for lan_wire:
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: off
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: off
tx-vlan-offload: off
ntuple-filters: off
receive-hashing: off

The Bridge device on the other hand....

Offload parameters for lan:
rx-checksumming: on
tx-checksumming: on
scatter-gather: off
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: off
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: off
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: off

Is it possible that the new code is tripped up by the bridge offload
parameters somehow and thinks that the gianfar nic supports it as
well?
As I wrote in my initial mail, connecting to the Bridge itself works,
I can also see ARP requests and Broadcasts, the rest just seems to
disappear.

As for the commits I reverted:
a4aeb2662 vlan: kill __vlan_hwaccel_rx and vlan_hwaccel_rx
6dacaddd4 vlan: kill vlan_hwaccel_receive_skb
7890a5b9c vlan: kill ndo_vlan_rx_register
b852b7208 gianfar: fix bug caused by 87c288c6e9aa31720b72e2bc2d665e24e1653c3e

87c288c6e gianfar: do vlan cleanup

I really think that there is only some small change in the gianfar
code needed, but I could not figure it out myself.
Please tell me if you need any more information.

Kind regards,
Michael Guntsche

Re: BUG: Bisected Gianfar in bridge not forwarding packets (was: 3.0-rc1 Bridge not forwarding unicast packages)

[Jiri Pirko]

Wed, Aug 10, 2011 at 02:50:20AM CEST, mguntsche@gmail.com wrote:
>Good evening/morning,
>
>> I just upgraded my router/bridge combo to 3.1-rc1 from 3.0 for
>> testing. On a first look everything seemed to work fine, but when I
>> tried to connect via openvpn to my internal network (tap0 being bridged
>> with the internal network) I noticed that I was not able to access the
>> server on my internal network. I could access the bridge (which is
>> acting as the openvpn server as well) just fine though.
>> To debug this I ran tcpdump on the openvpn client and started a ping to the
>> internal network. I could see the ARP requests being answered.....
><snip>
>
>After much digging and testing I found the commit that causes the
>problem for me here.
>
>87c288c6e gianfar: do vlan cleanup
>
>Before this commit my bridge works fine. I simplified the test case
>and just tried to access the server from a local wireless device with
>gets also bridged via
>a gianfar nic to the wired lan. Before commit 87c288c6e I can ping the
>device from the internal network afterwards it stops.
>For testing purposes I reverted this commit (plus some others to make
>the code compile) and it started working again.
>The hardware is a routerboard with three onboard nics two of em gianfar.
>The gianfar nics do not support any kind of offloading,

Thanks for testing/report. I'm not sure I understand your simplified
case :(

Anyway, gianfar supports rx/tx vlan accel. Would you please try to turn
it on/off manually by ethtool to see if it works in some setup?

I presume you have no vlans involved in your setup, right?
	
>
>Offload parameters for lan_wire:
>rx-checksumming: off
>tx-checksumming: off
>scatter-gather: off
>tcp-segmentation-offload: off
>udp-fragmentation-offload: off
>generic-segmentation-offload: off
>generic-receive-offload: on
>large-receive-offload: off
>rx-vlan-offload: off
>tx-vlan-offload: off
>ntuple-filters: off
>receive-hashing: off
>
>The Bridge device on the other hand....
>
>Offload parameters for lan:
>rx-checksumming: on
>tx-checksumming: on
>scatter-gather: off
>tcp-segmentation-offload: off
>udp-fragmentation-offload: off
>generic-segmentation-offload: off
>generic-receive-offload: on
>large-receive-offload: off
>rx-vlan-offload: off
>tx-vlan-offload: on
>ntuple-filters: off
>receive-hashing: off
>
>Is it possible that the new code is tripped up by the bridge offload
>parameters somehow and thinks that the gianfar nic supports it as
>well?

I don't see how now. Outside the bridge all is working correctly?

>As I wrote in my initial mail, connecting to the Bridge itself works,
>I can also see ARP requests and Broadcasts, the rest just seems to
>disappear.
>
>As for the commits I reverted:
>a4aeb2662 vlan: kill __vlan_hwaccel_rx and vlan_hwaccel_rx
>6dacaddd4 vlan: kill vlan_hwaccel_receive_skb
>7890a5b9c vlan: kill ndo_vlan_rx_register
>b852b7208 gianfar: fix bug caused by 87c288c6e9aa31720b72e2bc2d665e24e1653c3e
>
>87c288c6e gianfar: do vlan cleanup
>
>I really think that there is only some small change in the gianfar
>code needed, but I could not figure it out myself.
>Please tell me if you need any more information.
>
>Kind regards,
>Michael Guntsche

Re: BUG: Bisected Gianfar in bridge not forwarding packets (was: 3.0-rc1 Bridge not forwarding unicast packages)

[Michael Guntsche]

On Wed, Aug 10, 2011 at 9:07 AM, Jiri Pirko <jpirko@redhat.com> wrote:
> Wed, Aug 10, 2011 at 02:50:20AM CEST, mguntsche@gmail.com wrote:
>>Good evening/morning,
>>
>>> I just upgraded my router/bridge combo to 3.1-rc1 from 3.0 for
>>> testing. On a first look everything seemed to work fine, but when I
>>> tried to connect via openvpn to my internal network (tap0 being bridged
>>> with the internal network) I noticed that I was not able to access the
>>> server on my internal network. I could access the bridge (which is
>>> acting as the openvpn server as well) just fine though.
>>> To debug this I ran tcpdump on the openvpn client and started a ping to the
>>> internal network. I could see the ARP requests being answered.....
>><snip>
>>
>>After much digging and testing I found the commit that causes the
>>problem for me here.
>>
>>87c288c6e gianfar: do vlan cleanup
>>
>>Before this commit my bridge works fine. I simplified the test case
>>and just tried to access the server from a local wireless device with
>>gets also bridged via
>>a gianfar nic to the wired lan. Before commit 87c288c6e I can ping the
>>device from the internal network afterwards it stops.
>>For testing purposes I reverted this commit (plus some others to make
>>the code compile) and it started working again.
>>The hardware is a routerboard with three onboard nics two of em gianfar.
>>The gianfar nics do not support any kind of offloading,
>
> Thanks for testing/report. I'm not sure I understand your simplified
> case :(
Sorry if my explanation was confusing but it was late night/morning
when I wrote it.

The setup is quite easy.

WLAN ----->|Bridge|<----- Wired Lan (gianfar nic)

Client (in WLAN): 192.168.42.55
Bridge: 192.168.42.1
Server (on Wired): 192.168.42.127

With your patch the Bridge can ping the client and the server, but the
client cannot reach the server and vice-versa.

>
> Anyway, gianfar supports rx/tx vlan accel. Would you please try to turn
> it on/off manually by ethtool to see if it works in some setup?
The gianfar chip in this routerboard does NOT support vlan accel it is
a TSEC model.
ethtool complains that this feature is not supported.

>
> I presume you have no vlans involved in your setup, right?
No, there are no vlans in my setup.


>>Is it possible that the new code is tripped up by the bridge offload
>>parameters somehow and thinks that the gianfar nic supports it as
>>well?
>
> I don't see how now. Outside the bridge all is working correctly?
Yes I can access the server from the bridge so the network is working.
For me it looks like that the packets are just dropped when they
should go through the bridge.

Kind regards,
Michael

Re: BUG: Bisected Gianfar in bridge not forwarding packets (was: 3.0-rc1 Bridge not forwarding unicast packages)

[Jiri Pirko]

Wed, Aug 10, 2011 at 09:40:50AM CEST, mguntsche@gmail.com wrote:
>On Wed, Aug 10, 2011 at 9:07 AM, Jiri Pirko <jpirko@redhat.com> wrote:
>> Wed, Aug 10, 2011 at 02:50:20AM CEST, mguntsche@gmail.com wrote:
>>>Good evening/morning,
>>>
>>>> I just upgraded my router/bridge combo to 3.1-rc1 from 3.0 for
>>>> testing. On a first look everything seemed to work fine, but when I
>>>> tried to connect via openvpn to my internal network (tap0 being bridged
>>>> with the internal network) I noticed that I was not able to access the
>>>> server on my internal network. I could access the bridge (which is
>>>> acting as the openvpn server as well) just fine though.
>>>> To debug this I ran tcpdump on the openvpn client and started a ping to the
>>>> internal network. I could see the ARP requests being answered.....
>>><snip>
>>>
>>>After much digging and testing I found the commit that causes the
>>>problem for me here.
>>>
>>>87c288c6e gianfar: do vlan cleanup
>>>
>>>Before this commit my bridge works fine. I simplified the test case
>>>and just tried to access the server from a local wireless device with
>>>gets also bridged via
>>>a gianfar nic to the wired lan. Before commit 87c288c6e I can ping the
>>>device from the internal network afterwards it stops.
>>>For testing purposes I reverted this commit (plus some others to make
>>>the code compile) and it started working again.
>>>The hardware is a routerboard with three onboard nics two of em gianfar.
>>>The gianfar nics do not support any kind of offloading,
>>
>> Thanks for testing/report. I'm not sure I understand your simplified
>> case :(
>Sorry if my explanation was confusing but it was late night/morning
>when I wrote it.
>
>The setup is quite easy.
>
>WLAN ----->|Bridge|<----- Wired Lan (gianfar nic)
>
>Client (in WLAN): 192.168.42.55
>Bridge: 192.168.42.1
>Server (on Wired): 192.168.42.127
>
>With your patch the Bridge can ping the client and the server, but the
>client cannot reach the server and vice-versa.

This is very interesting. Can you please find out of packets get lost on
tx or rx of gianfar? Please try to watch the trafic by tcpdump on bridge
and on gianfar device.


>
>>
>> Anyway, gianfar supports rx/tx vlan accel. Would you please try to turn
>> it on/off manually by ethtool to see if it works in some setup?
>The gianfar chip in this routerboard does NOT support vlan accel it is
>a TSEC model.
>ethtool complains that this feature is not supported.

What routerboard is it? Maybe I can get one to see this myself. Thanks.


Can you please send me output of following command?
"cat /sys/class/net/[gianfar devname]/features"

Also, please try with following testing patch:


diff --git a/drivers/net/gianfar.c b/drivers/net/gianfar.c
index 2659daa..2727bb7 100644
--- a/drivers/net/gianfar.c
+++ b/drivers/net/gianfar.c
@@ -2711,8 +2711,8 @@ static int gfar_process_frame(struct net_device *dev, struct sk_buff *skb,
 	skb->protocol = eth_type_trans(skb, dev);
 
 	/* Set vlan tag */
-	if (fcb->flags & RXFCB_VLN)
-		__vlan_hwaccel_put_tag(skb, fcb->vlctl);
+	//if (fcb->flags & RXFCB_VLN)
+	//	__vlan_hwaccel_put_tag(skb, fcb->vlctl);
 
 	/* Send the packet up the stack */
 	ret = netif_receive_skb(skb);

>
>>
>> I presume you have no vlans involved in your setup, right?
>No, there are no vlans in my setup.
>
>
>>>Is it possible that the new code is tripped up by the bridge offload
>>>parameters somehow and thinks that the gianfar nic supports it as
>>>well?
>>
>> I don't see how now. Outside the bridge all is working correctly?
>Yes I can access the server from the bridge so the network is working.
>For me it looks like that the packets are just dropped when they
>should go through the bridge.
>
>Kind regards,
>Michael

Re: BUG: Bisected Gianfar in bridge not forwarding packets (was: 3.0-rc1 Bridge not forwarding unicast packages)

[Michael Guntsche]

On Wed, Aug 10, 2011 at 10:37 AM, Jiri Pirko <jpirko@redhat.com> wrote:
> Wed, Aug 10, 2011 at 09:40:50AM CEST, mguntsche@gmail.com wrote:
>>On Wed, Aug 10, 2011 at 9:07 AM, Jiri Pirko <jpirko@redhat.com> wrote:
>>> Wed, Aug 10, 2011 at 02:50:20AM CEST, mguntsche@gmail.com wrote:
>>>>Good evening/morning,
>>>>
>>>>> I just upgraded my router/bridge combo to 3.1-rc1 from 3.0 for
>>>>> testing. On a first look everything seemed to work fine, but when I
>>>>> tried to connect via openvpn to my internal network (tap0 being bridged
>>>>> with the internal network) I noticed that I was not able to access the
>>>>> server on my internal network. I could access the bridge (which is
>>>>> acting as the openvpn server as well) just fine though.
>>>>> To debug this I ran tcpdump on the openvpn client and started a ping to the
>>>>> internal network. I could see the ARP requests being answered.....
>>>><snip>
>>>>
>>>>After much digging and testing I found the commit that causes the
>>>>problem for me here.
>>>>
>>>>87c288c6e gianfar: do vlan cleanup
>>>>
>>>>Before this commit my bridge works fine. I simplified the test case
>>>>and just tried to access the server from a local wireless device with
>>>>gets also bridged via
>>>>a gianfar nic to the wired lan. Before commit 87c288c6e I can ping the
>>>>device from the internal network afterwards it stops.
>>>>For testing purposes I reverted this commit (plus some others to make
>>>>the code compile) and it started working again.
>>>>The hardware is a routerboard with three onboard nics two of em gianfar.
>>>>The gianfar nics do not support any kind of offloading,
>>>
>>> Thanks for testing/report. I'm not sure I understand your simplified
>>> case :(
>>Sorry if my explanation was confusing but it was late night/morning
>>when I wrote it.
>>
>>The setup is quite easy.
>>
>>WLAN ----->|Bridge|<----- Wired Lan (gianfar nic)
>>
>>Client (in WLAN): 192.168.42.55
>>Bridge: 192.168.42.1
>>Server (on Wired): 192.168.42.127
>>
>>With your patch the Bridge can ping the client and the server, but the
>>client cannot reach the server and vice-versa.
>
> This is very interesting. Can you please find out of packets get lost on
> tx or rx of gianfar? Please try to watch the trafic by tcpdump on bridge
> and on gianfar device.
I can try to do that this afternoon.

>>> Anyway, gianfar supports rx/tx vlan accel. Would you please try to turn
>>> it on/off manually by ethtool to see if it works in some setup?
>>The gianfar chip in this routerboard does NOT support vlan accel it is
>>a TSEC model.
>>ethtool complains that this feature is not supported.
>
> What routerboard is it? Maybe I can get one to see this myself. Thanks.
The model is a Rb600a which is no longer available

> Can you please send me output of following command?
> "cat /sys/class/net/[gianfar devname]/features"
There is no file called features under the gianfar devname

>
> Also, please try with following testing patch:
>
>
> diff --git a/drivers/net/gianfar.c b/drivers/net/gianfar.c
> index 2659daa..2727bb7 100644
> --- a/drivers/net/gianfar.c
> +++ b/drivers/net/gianfar.c
> @@ -2711,8 +2711,8 @@ static int gfar_process_frame(struct net_device *dev, struct sk_buff *skb,
>        skb->protocol = eth_type_trans(skb, dev);
>
>        /* Set vlan tag */
> -       if (fcb->flags & RXFCB_VLN)
> -               __vlan_hwaccel_put_tag(skb, fcb->vlctl);
> +       //if (fcb->flags & RXFCB_VLN)
> +       //      __vlan_hwaccel_put_tag(skb, fcb->vlctl);
>
>        /* Send the packet up the stack */
>        ret = netif_receive_skb(skb);
Will do.

Kind regards,
Michael

Re: BUG: Bisected Gianfar in bridge not forwarding packets (was: 3.0-rc1 Bridge not forwarding unicast packages)

[Jiri Pirko]

Wed, Aug 10, 2011 at 02:50:20AM CEST, mguntsche@gmail.com wrote:
>Good evening/morning,
>
>> I just upgraded my router/bridge combo to 3.1-rc1 from 3.0 for
>> testing. On a first look everything seemed to work fine, but when I
>> tried to connect via openvpn to my internal network (tap0 being bridged
>> with the internal network) I noticed that I was not able to access the
>> server on my internal network. I could access the bridge (which is
>> acting as the openvpn server as well) just fine though.
>> To debug this I ran tcpdump on the openvpn client and started a ping to the
>> internal network. I could see the ARP requests being answered.....
><snip>
>
>After much digging and testing I found the commit that causes the
>problem for me here.
>
>87c288c6e gianfar: do vlan cleanup
>
>Before this commit my bridge works fine. I simplified the test case
>and just tried to access the server from a local wireless device with
>gets also bridged via
>a gianfar nic to the wired lan. Before commit 87c288c6e I can ping the
>device from the internal network afterwards it stops.
>For testing purposes I reverted this commit (plus some others to make
>the code compile) and it started working again.
>The hardware is a routerboard with three onboard nics two of em gianfar.
>The gianfar nics do not support any kind of offloading,
>
>Offload parameters for lan_wire:
>rx-checksumming: off
>tx-checksumming: off
>scatter-gather: off
>tcp-segmentation-offload: off
>udp-fragmentation-offload: off
>generic-segmentation-offload: off
>generic-receive-offload: on
>large-receive-offload: off
>rx-vlan-offload: off
>tx-vlan-offload: off
>ntuple-filters: off
>receive-hashing: off
>
>The Bridge device on the other hand....
>
>Offload parameters for lan:
>rx-checksumming: on
>tx-checksumming: on
>scatter-gather: off
>tcp-segmentation-offload: off
>udp-fragmentation-offload: off
>generic-segmentation-offload: off
>generic-receive-offload: on
>large-receive-offload: off
>rx-vlan-offload: off
>tx-vlan-offload: on
  ^^^^^^^^^^^^^^^^^^^ is this gfar?
>ntuple-filters: off
>receive-hashing: off

>
>Is it possible that the new code is tripped up by the bridge offload
>parameters somehow and thinks that the gianfar nic supports it as
>well?
>As I wrote in my initial mail, connecting to the Bridge itself works,
>I can also see ARP requests and Broadcasts, the rest just seems to
>disappear.
>
>As for the commits I reverted:
>a4aeb2662 vlan: kill __vlan_hwaccel_rx and vlan_hwaccel_rx
>6dacaddd4 vlan: kill vlan_hwaccel_receive_skb
>7890a5b9c vlan: kill ndo_vlan_rx_register
>b852b7208 gianfar: fix bug caused by 87c288c6e9aa31720b72e2bc2d665e24e1653c3e
>
>87c288c6e gianfar: do vlan cleanup
>
>I really think that there is only some small change in the gianfar
>code needed, but I could not figure it out myself.
>Please tell me if you need any more information.
>
>Kind regards,
>Michael Guntsche

Re: BUG: Bisected Gianfar in bridge not forwarding packets (was: 3.0-rc1 Bridge not forwarding unicast packages)

[Michael Guntsche]

On Wed, Aug 10, 2011 at 11:00 AM, Jiri Pirko <jpirko@redhat.com> wrote:
> Wed, Aug 10, 2011 at 02:50:20AM CEST, mguntsche@gmail.com wrote:
>>Good evening/morning,
>>
>>> I just upgraded my router/bridge combo to 3.1-rc1 from 3.0 for
>>> testing. On a first look everything seemed to work fine, but when I
>>> tried to connect via openvpn to my internal network (tap0 being bridged
>>> with the internal network) I noticed that I was not able to access the
>>> server on my internal network. I could access the bridge (which is
>>> acting as the openvpn server as well) just fine though.
>>> To debug this I ran tcpdump on the openvpn client and started a ping to the
>>> internal network. I could see the ARP requests being answered.....
>><snip>
>>
>>After much digging and testing I found the commit that causes the
>>problem for me here.
>>
>>87c288c6e gianfar: do vlan cleanup
>>
>>Before this commit my bridge works fine. I simplified the test case
>>and just tried to access the server from a local wireless device with
>>gets also bridged via
>>a gianfar nic to the wired lan. Before commit 87c288c6e I can ping the
>>device from the internal network afterwards it stops.
>>For testing purposes I reverted this commit (plus some others to make
>>the code compile) and it started working again.
>>The hardware is a routerboard with three onboard nics two of em gianfar.
>>The gianfar nics do not support any kind of offloading,
>>
>>Offload parameters for lan_wire:
>>rx-checksumming: off
>>tx-checksumming: off
>>scatter-gather: off
>>tcp-segmentation-offload: off
>>udp-fragmentation-offload: off
>>generic-segmentation-offload: off
>>generic-receive-offload: on
>>large-receive-offload: off
>>rx-vlan-offload: off
>>tx-vlan-offload: off
>>ntuple-filters: off
>>receive-hashing: off
>>
>>The Bridge device on the other hand....
>>
>>Offload parameters for lan:
>>rx-checksumming: on
>>tx-checksumming: on
>>scatter-gather: off
>>tcp-segmentation-offload: off
>>udp-fragmentation-offload: off
>>generic-segmentation-offload: off
>>generic-receive-offload: on
>>large-receive-offload: off
>>rx-vlan-offload: off
>>tx-vlan-offload: on
>  ^^^^^^^^^^^^^^^^^^^ is this gfar?
No the "Lan" nic is the bridge itself. The gfar in question is lan_wire.

/Michael

Re: BUG: Bisected Gianfar in bridge not forwarding packets (was: 3.0-rc1 Bridge not forwarding unicast packages)

[Michael Guntsche]

On Wed, Aug 10, 2011 at 11:59 AM, Michael Guntsche <mguntsche@gmail.com> wrote:
<snip>
>>>Offload parameters for lan_wire:
>>>rx-checksumming: off
>>>tx-checksumming: off
>>>scatter-gather: off
>>>tcp-segmentation-offload: off
>>>udp-fragmentation-offload: off
>>>generic-segmentation-offload: off
>>>generic-receive-offload: on
>>>large-receive-offload: off
>>>rx-vlan-offload: off
>>>tx-vlan-offload: off
>>>ntuple-filters: off
>>>receive-hashing: off
>>>
>>>The Bridge device on the other hand....
>>>
>>>Offload parameters for lan:
>>>rx-checksumming: on
>>>tx-checksumming: on
>>>scatter-gather: off
>>>tcp-segmentation-offload: off
>>>udp-fragmentation-offload: off
>>>generic-segmentation-offload: off
>>>generic-receive-offload: on
>>>large-receive-offload: off
>>>rx-vlan-offload: off
>>>tx-vlan-offload: on
>>  ^^^^^^^^^^^^^^^^^^^ is this gfar?
> No the "Lan" nic is the bridge itself. The gfar in question is lan_wire.
>
> /Michael
>

Ok I would have saved hours of bisecting if I had just used the -e
switch with tcpdump from the beginning.
Jiri first of all the patch makes the connection work again. I can
ping the client on the wlan from the server and vice-versa. Taking a
look at the tcpdump (WITH -e) makes it obvious why it fails with the
non patched version...

This is a capture from the gfar lan port on the bridge with no patch applied
12:13:24.011492 00:13:d4:4f:a2:dc (oui Unknown) > b4:07:f9:70:b7:c1
(oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 19, p 0,
ethertype IPv4, gibson.comsick.at > 192.168.42.55: ICMP echo request,
id 23567, seq 74, length 64

As you can see we get a VLAN package????

With the patched version:
12:16:33.756966 00:13:d4:4f:a2:dc (oui Unknown) > b4:07:f9:70:b7:c1
(oui Unknown), ethertype IPv4 (0x0800), length 98: gibson.comsick.at >
192.168.42.55: ICMP echo request, id 23669, seq 2, length 64
12:16:33.843289 b4:07:f9:70:b7:c1 (oui Unknown) > 00:13:d4:4f:a2:dc
(oui Unknown), ethertype IPv4 (0x0800), length 98: 192.168.42.55 >
gibson.comsick.at: ICMP echo reply, id 23669, seq 2, length 64

No VLAN and I get my replies.
All that is left to do now is to find out why the current driver
thinks that it is on a VLAN when it isn't.

Any ideas Jiri?

Kind regards,
Michael

Re: BUG: Bisected Gianfar in bridge not forwarding packets (was: 3.0-rc1 Bridge not forwarding unicast packages)

[Jiri Pirko]

Wed, Aug 10, 2011 at 12:30:16PM CEST, mguntsche@gmail.com wrote:
>On Wed, Aug 10, 2011 at 11:59 AM, Michael Guntsche <mguntsche@gmail.com> wrote:
><snip>
>>>>Offload parameters for lan_wire:
>>>>rx-checksumming: off
>>>>tx-checksumming: off
>>>>scatter-gather: off
>>>>tcp-segmentation-offload: off
>>>>udp-fragmentation-offload: off
>>>>generic-segmentation-offload: off
>>>>generic-receive-offload: on
>>>>large-receive-offload: off
>>>>rx-vlan-offload: off
>>>>tx-vlan-offload: off
>>>>ntuple-filters: off
>>>>receive-hashing: off
>>>>
>>>>The Bridge device on the other hand....
>>>>
>>>>Offload parameters for lan:
>>>>rx-checksumming: on
>>>>tx-checksumming: on
>>>>scatter-gather: off
>>>>tcp-segmentation-offload: off
>>>>udp-fragmentation-offload: off
>>>>generic-segmentation-offload: off
>>>>generic-receive-offload: on
>>>>large-receive-offload: off
>>>>rx-vlan-offload: off
>>>>tx-vlan-offload: on
>>>  ^^^^^^^^^^^^^^^^^^^ is this gfar?
>> No the "Lan" nic is the bridge itself. The gfar in question is lan_wire.
>>
>> /Michael
>>
>
>Ok I would have saved hours of bisecting if I had just used the -e
>switch with tcpdump from the beginning.
>Jiri first of all the patch makes the connection work again. I can
>ping the client on the wlan from the server and vice-versa. Taking a
>look at the tcpdump (WITH -e) makes it obvious why it fails with the
>non patched version...
>
>This is a capture from the gfar lan port on the bridge with no patch applied
>12:13:24.011492 00:13:d4:4f:a2:dc (oui Unknown) > b4:07:f9:70:b7:c1
>(oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 19, p 0,
>ethertype IPv4, gibson.comsick.at > 192.168.42.55: ICMP echo request,
>id 23567, seq 74, length 64
>
>As you can see we get a VLAN package????

Ugh, this is what I expected. Patch to fix:

Subject: [patch net-2.6] gianfar: prevent buggy hw rx vlan tagging

On some buggy chips, "vlan tag present" flag is set which causes packet
loss. Fix this by checking if rx vlan accel is enabled in features.

Reported-by: Michael Guntsche <mguntsche@gmail.com>
Signed-off-by: Jiri Pirko <jpirko@redhat.com>
---
 drivers/net/gianfar.c |    9 +++++++--
 1 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/net/gianfar.c b/drivers/net/gianfar.c
index 2659daa..31d5c57 100644
--- a/drivers/net/gianfar.c
+++ b/drivers/net/gianfar.c
@@ -2710,8 +2710,13 @@ static int gfar_process_frame(struct net_device *dev, struct sk_buff *skb,
 	/* Tell the skb what kind of packet this is */
 	skb->protocol = eth_type_trans(skb, dev);
 
-	/* Set vlan tag */
-	if (fcb->flags & RXFCB_VLN)
+	/*
+	 * There's need to check for NETIF_F_HW_VLAN_RX here.
+	 * Even if vlan rx accel is disabled, on some chips
+	 * RXFCB_VLN is pseudo randomly set.
+	 */
+	if (dev->features & NETIF_F_HW_VLAN_RX &&
+	    fcb->flags & RXFCB_VLN)
 		__vlan_hwaccel_put_tag(skb, fcb->vlctl);
 
 	/* Send the packet up the stack */
-- 
1.7.6

Re: BUG: Bisected Gianfar in bridge not forwarding packets (was: 3.0-rc1 Bridge not forwarding unicast packages)

[Michael Guntsche]

On Wed, Aug 10, 2011 at 1:19 PM, Jiri Pirko <jpirko@redhat.com> wrote:
> Wed, Aug 10, 2011 at 12:30:16PM CEST, mguntsche@gmail.com wrote:
>>On Wed, Aug 10, 2011 at 11:59 AM, Michael Guntsche <mguntsche@gmail.com> wrote:
>><snip>
>>>>>Offload parameters for lan_wire:
>>>>>rx-checksumming: off
>>>>>tx-checksumming: off
>>>>>scatter-gather: off
>>>>>tcp-segmentation-offload: off
>>>>>udp-fragmentation-offload: off
>>>>>generic-segmentation-offload: off
>>>>>generic-receive-offload: on
>>>>>large-receive-offload: off
>>>>>rx-vlan-offload: off
>>>>>tx-vlan-offload: off
>>>>>ntuple-filters: off
>>>>>receive-hashing: off
>>>>>
>>>>>The Bridge device on the other hand....
>>>>>
>>>>>Offload parameters for lan:
>>>>>rx-checksumming: on
>>>>>tx-checksumming: on
>>>>>scatter-gather: off
>>>>>tcp-segmentation-offload: off
>>>>>udp-fragmentation-offload: off
>>>>>generic-segmentation-offload: off
>>>>>generic-receive-offload: on
>>>>>large-receive-offload: off
>>>>>rx-vlan-offload: off
>>>>>tx-vlan-offload: on
>>>>  ^^^^^^^^^^^^^^^^^^^ is this gfar?
>>> No the "Lan" nic is the bridge itself. The gfar in question is lan_wire.
>>>
>>> /Michael
>>>
>>
>>Ok I would have saved hours of bisecting if I had just used the -e
>>switch with tcpdump from the beginning.
>>Jiri first of all the patch makes the connection work again. I can
>>ping the client on the wlan from the server and vice-versa. Taking a
>>look at the tcpdump (WITH -e) makes it obvious why it fails with the
>>non patched version...
>>
>>This is a capture from the gfar lan port on the bridge with no patch applied
>>12:13:24.011492 00:13:d4:4f:a2:dc (oui Unknown) > b4:07:f9:70:b7:c1
>>(oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 19, p 0,
>>ethertype IPv4, gibson.comsick.at > 192.168.42.55: ICMP echo request,
>>id 23567, seq 74, length 64
>>
>>As you can see we get a VLAN package????
>
> Ugh, this is what I expected. Patch to fix:
>
> Subject: [patch net-2.6] gianfar: prevent buggy hw rx vlan tagging
>
> On some buggy chips, "vlan tag present" flag is set which causes packet
> loss. Fix this by checking if rx vlan accel is enabled in features.
>
> Reported-by: Michael Guntsche <mguntsche@gmail.com>
> Signed-off-by: Jiri Pirko <jpirko@redhat.com>
> ---
>  drivers/net/gianfar.c |    9 +++++++--
>  1 files changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/net/gianfar.c b/drivers/net/gianfar.c
> index 2659daa..31d5c57 100644
> --- a/drivers/net/gianfar.c
> +++ b/drivers/net/gianfar.c
> @@ -2710,8 +2710,13 @@ static int gfar_process_frame(struct net_device *dev, struct sk_buff *skb,
>        /* Tell the skb what kind of packet this is */
>        skb->protocol = eth_type_trans(skb, dev);
>
> -       /* Set vlan tag */
> -       if (fcb->flags & RXFCB_VLN)
> +       /*
> +        * There's need to check for NETIF_F_HW_VLAN_RX here.
> +        * Even if vlan rx accel is disabled, on some chips
> +        * RXFCB_VLN is pseudo randomly set.
> +        */
> +       if (dev->features & NETIF_F_HW_VLAN_RX &&
> +           fcb->flags & RXFCB_VLN)
>                __vlan_hwaccel_put_tag(skb, fcb->vlctl);
>
>        /* Send the packet up the stack */
> --
> 1.7.6
>
>

Jiri, there seems to be another bug lingering in the bridge code,
which might cause this problem in the first place but I am not really
sure. I looked at the ethtool output some more and I noticed that some
features were enabled on the Bridge which should be off.

With 3.1-rc1
Offload parameters for lan (This is the bridge interface itself):
rx-checksumming: on                  <---- ON
tx-checksumming: on                  <---- ON
scatter-gather: off
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: off
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: off
tx-vlan-offload: on                         <---- ON

I booted an older kernel on the same hardware (2.6.39) and the output differs.

Offload parameters for lan:
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
rx-vlan-offload: off
tx-vlan-offload: off
ntuple-filters: off
receive-hashing: off

As you can see all the values are OFF which is correct, since no
attached interface supports these features.
Is it possible that this is tripping up the now changed gianfar code somehow?
What I am trying to say. Could it be that your patch is hiding a
problem that is now surfacing with a change in the bridge code?

I am thinking about this commit:
c4d27ef95:  bridge: convert br_features_recompute() to ndo_fix_features

Of course this could also be a totally separate bug as well.

Any thoughts, Dave, Stephen?

Kind regards,
Michael Guntsche

Re: BUG: Bisected Gianfar in bridge not forwarding packets (was: 3.0-rc1 Bridge not forwarding unicast packages)

[Jiri Pirko]

Wed, Aug 10, 2011 at 01:51:54PM CEST, mguntsche@gmail.com wrote:
>On Wed, Aug 10, 2011 at 1:19 PM, Jiri Pirko <jpirko@redhat.com> wrote:
>> Wed, Aug 10, 2011 at 12:30:16PM CEST, mguntsche@gmail.com wrote:
>>>On Wed, Aug 10, 2011 at 11:59 AM, Michael Guntsche <mguntsche@gmail.com> wrote:
>>><snip>
>>>>>>Offload parameters for lan_wire:
>>>>>>rx-checksumming: off
>>>>>>tx-checksumming: off
>>>>>>scatter-gather: off
>>>>>>tcp-segmentation-offload: off
>>>>>>udp-fragmentation-offload: off
>>>>>>generic-segmentation-offload: off
>>>>>>generic-receive-offload: on
>>>>>>large-receive-offload: off
>>>>>>rx-vlan-offload: off
>>>>>>tx-vlan-offload: off
>>>>>>ntuple-filters: off
>>>>>>receive-hashing: off
>>>>>>
>>>>>>The Bridge device on the other hand....
>>>>>>
>>>>>>Offload parameters for lan:
>>>>>>rx-checksumming: on
>>>>>>tx-checksumming: on
>>>>>>scatter-gather: off
>>>>>>tcp-segmentation-offload: off
>>>>>>udp-fragmentation-offload: off
>>>>>>generic-segmentation-offload: off
>>>>>>generic-receive-offload: on
>>>>>>large-receive-offload: off
>>>>>>rx-vlan-offload: off
>>>>>>tx-vlan-offload: on
>>>>>  ^^^^^^^^^^^^^^^^^^^ is this gfar?
>>>> No the "Lan" nic is the bridge itself. The gfar in question is lan_wire.
>>>>
>>>> /Michael
>>>>
>>>
>>>Ok I would have saved hours of bisecting if I had just used the -e
>>>switch with tcpdump from the beginning.
>>>Jiri first of all the patch makes the connection work again. I can
>>>ping the client on the wlan from the server and vice-versa. Taking a
>>>look at the tcpdump (WITH -e) makes it obvious why it fails with the
>>>non patched version...
>>>
>>>This is a capture from the gfar lan port on the bridge with no patch applied
>>>12:13:24.011492 00:13:d4:4f:a2:dc (oui Unknown) > b4:07:f9:70:b7:c1
>>>(oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 19, p 0,
>>>ethertype IPv4, gibson.comsick.at > 192.168.42.55: ICMP echo request,
>>>id 23567, seq 74, length 64
>>>
>>>As you can see we get a VLAN package????
>>
>> Ugh, this is what I expected. Patch to fix:
>>
>> Subject: [patch net-2.6] gianfar: prevent buggy hw rx vlan tagging
>>
>> On some buggy chips, "vlan tag present" flag is set which causes packet
>> loss. Fix this by checking if rx vlan accel is enabled in features.
>>
>> Reported-by: Michael Guntsche <mguntsche@gmail.com>
>> Signed-off-by: Jiri Pirko <jpirko@redhat.com>
>> ---
>>  drivers/net/gianfar.c |    9 +++++++--
>>  1 files changed, 7 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/net/gianfar.c b/drivers/net/gianfar.c
>> index 2659daa..31d5c57 100644
>> --- a/drivers/net/gianfar.c
>> +++ b/drivers/net/gianfar.c
>> @@ -2710,8 +2710,13 @@ static int gfar_process_frame(struct net_device *dev, struct sk_buff *skb,
>>        /* Tell the skb what kind of packet this is */
>>        skb->protocol = eth_type_trans(skb, dev);
>>
>> -       /* Set vlan tag */
>> -       if (fcb->flags & RXFCB_VLN)
>> +       /*
>> +        * There's need to check for NETIF_F_HW_VLAN_RX here.
>> +        * Even if vlan rx accel is disabled, on some chips
>> +        * RXFCB_VLN is pseudo randomly set.
>> +        */
>> +       if (dev->features & NETIF_F_HW_VLAN_RX &&
>> +           fcb->flags & RXFCB_VLN)
>>                __vlan_hwaccel_put_tag(skb, fcb->vlctl);
>>
>>        /* Send the packet up the stack */
>> --
>> 1.7.6
>>
>>
>
>Jiri, there seems to be another bug lingering in the bridge code,
>which might cause this problem in the first place but I am not really
>sure. I looked at the ethtool output some more and I noticed that some
>features were enabled on the Bridge which should be off.
>
>With 3.1-rc1
>Offload parameters for lan (This is the bridge interface itself):
>rx-checksumming: on                  <---- ON
>tx-checksumming: on                  <---- ON
>scatter-gather: off
>tcp-segmentation-offload: off
>udp-fragmentation-offload: off
>generic-segmentation-offload: off
>generic-receive-offload: on
>large-receive-offload: off
>rx-vlan-offload: off
>tx-vlan-offload: on                         <---- ON
>
>I booted an older kernel on the same hardware (2.6.39) and the output differs.
>
>Offload parameters for lan:
>rx-checksumming: off
>tx-checksumming: off
>scatter-gather: off
>tcp-segmentation-offload: off
>udp-fragmentation-offload: off
>generic-segmentation-offload: off
>generic-receive-offload: off
>large-receive-offload: off
>rx-vlan-offload: off
>tx-vlan-offload: off
>ntuple-filters: off
>receive-hashing: off
>
>As you can see all the values are OFF which is correct, since no
>attached interface supports these features.
>Is it possible that this is tripping up the now changed gianfar code somehow?
>What I am trying to say. Could it be that your patch is hiding a
>problem that is now surfacing with a change in the bridge code?

Well this seems unrelated. The problem is that gianfar hw is indicating
it received vlan tagged packet even in case it did not.

>
>I am thinking about this commit:
>c4d27ef95:  bridge: convert br_features_recompute() to ndo_fix_features
>
>Of course this could also be a totally separate bug as well.
>
>Any thoughts, Dave, Stephen?
>
>Kind regards,
>Michael Guntsche

Re: BUG: Bisected Gianfar in bridge not forwarding packets (was: 3.0-rc1 Bridge not forwarding unicast packages)

[Ben Hutchings]

On Wed, 2011-08-10 at 13:51 +0200, Michael Guntsche wrote:
[...]
> Jiri, there seems to be another bug lingering in the bridge code,
> which might cause this problem in the first place but I am not really
> sure. I looked at the ethtool output some more and I noticed that some
> features were enabled on the Bridge which should be off.
[...]

That's an intentional change.  There are software fallbacks for TX
checksumming and VLAN tag insertion.  Bridge devices (and some other
software devices) now always advertise them so that the fallback is not
used until and unless it is known that the real output device doesn't
support the feature.

Ben.

-- 
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

Re: BUG: Bisected Gianfar in bridge not forwarding packets

[David Miller]

From: Jiri Pirko <jpirko@redhat.com>
Date: Wed, 10 Aug 2011 13:19:48 +0200

> Subject: [patch net-2.6] gianfar: prevent buggy hw rx vlan tagging
> 
> On some buggy chips, "vlan tag present" flag is set which causes packet
> loss. Fix this by checking if rx vlan accel is enabled in features.
> 
> Reported-by: Michael Guntsche <mguntsche@gmail.com>
> Signed-off-by: Jiri Pirko <jpirko@redhat.com>

Applied.