r8169 doing more work than napi weight

r8169 doing more work than napi weight

[Timo Teras]

Hi,

I'm getting:

WARNING: at linux-grsec/src/linux-3.4/net/core/dev.c:3875 net_rx_action+0xab/0x153()

on one of my r8169 boxes.

This would be the:
	WARN_ON_ONCE(work > weight); 

Now the only way this seems to be possible to happen is that the AMD
workaround triggers:
        if ((desc->opts2 & cpu_to_le32(0xfffe000)) &&
            (tp->mac_version == RTL_GIGA_MAC_VER_05)) {
                 desc->opts2 = 0;
                 cur_rx++;
        }

And yes, the hardware where the WARN_ON_ONCE triggers is indeed
RTL_GIGA_MAC_VER_05.

This would cause cur_rx to be incremented twice in the loop, but
rx_left not decremented accordingly.

As the work done is counted finally based on cur_rx, we might end up
returning more work done than what was our quota.

This has also the unwanted consequence of messing NAPI state as if more
work than quota was done then polling is stopped as the work == weight
does not trigger and the polling is not rescheduled.

Git log says that this workaround was copied from Realtek's r8168
driver, but I don't see anything like this there anymore.

I'm wondering if we should just delete the
	cur_rx++;
Or add:
	rx_left--;

Or just delete the whole block as obsolete. 'git log' says the problem
should have gone away by always using hardware Rx VLAN. See commit
05af214 "r8169: fix Ethernet Hangup for RTL8110SC rev d".

Thanks,
 Timo

Re: r8169 doing more work than napi weight

[Francois Romieu]

Timo Teras <timo.teras@iki.fi> :
[...]
> Or just delete the whole block as obsolete. 'git log' says the problem
> should have gone away by always using hardware Rx VLAN. See commit
> 05af214 "r8169: fix Ethernet Hangup for RTL8110SC rev d".

Yes, this stuff looks really strange.

Turning the whole "(desc->opts2 & cpu_to_le32(0xfffe000)) && ... " test
as a WARN_ON_ONCE in the RxRES branch seems a good trade-off from a
coward's viewpoint.

-- 
Ueimor

[PATCH net] r8169: remove the obsolete and incorrect AMD workaround

[Timo Teräs]

This was introduced in commit 6dccd16 "r8169: merge with version
6.001.00 of Realtek's r8169 driver". I did not find the version
6.001.00 online, but in 6.002.00 or any later r8169 from Realtek
this hunk is no longer present.

Also commit 05af214 "r8169: fix Ethernet Hangup for RTL8110SC
rev d" claims to have fixed this issue otherwise.

The magic compare mask of 0xfffe000 is dubious as it masks
parts of the Reserved part, and parts of the VLAN tag. But this
does not make much sense as the VLAN tag parts are perfectly
valid there. In matter of fact this seems to be triggered with
any VLAN tagged packet as RxVlanTag bit is matched. I would
suspect 0xfffe0000 was intended to test reserved part only.

Finally, this hunk is evil as it can cause more packets to be
handled than what was NAPI quota causing net/core/dev.c:
net_rx_action(): WARN_ON_ONCE(work > weight) to trigger, and
mess up the NAPI state causing device to hang.

As result, any system using VLANs and having high receive
traffic (so that NAPI poll budget limits rtl_rx) would result
in device hang.

Signed-off-by: Timo Teräs <timo.teras@iki.fi>
---
I decided to suggest just removal without the suggested WARN_ON_ONCE.
This is because the test is incorrect as explained above, and not
present anywhere else either. Oh yeah, and I'm feeling brave today ;)

If acceptable, this should go to -stable too. The hunk was introduced
in v2.6.23-rc1 and the alternate fix went in to v2.6.32-rc6.

 drivers/net/ethernet/realtek/r8169.c | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/drivers/net/ethernet/realtek/r8169.c b/drivers/net/ethernet/realtek/r8169.c
index ed96f30..c28bc31 100644
--- a/drivers/net/ethernet/realtek/r8169.c
+++ b/drivers/net/ethernet/realtek/r8169.c
@@ -6111,13 +6111,6 @@ process_pkt:
 			tp->rx_stats.bytes += pkt_size;
 			u64_stats_update_end(&tp->rx_stats.syncp);
 		}
-
-		/* Work around for AMD plateform. */
-		if ((desc->opts2 & cpu_to_le32(0xfffe000)) &&
-		    (tp->mac_version == RTL_GIGA_MAC_VER_05)) {
-			desc->opts2 = 0;
-			cur_rx++;
-		}
 	}
 
 	count = cur_rx - tp->cur_rx;
-- 
1.8.1

Re: [PATCH net] r8169: remove the obsolete and incorrect AMD workaround

[Francois Romieu]

Timo Teräs <timo.teras@iki.fi> :
[...]

Acked-by: Francois Romieu <romieu@fr.zoreil.com>

desc->opts2 is now a bit different in the RxRes path - the only path
where the patch can be noticed. I hope it won't uncloak something else.

-- 
Ueimor

Re: [PATCH net] r8169: remove the obsolete and incorrect AMD workaround

[David Miller]

From: Francois Romieu <romieu@fr.zoreil.com>
Date: Wed, 23 Jan 2013 01:05:41 +0100

> Timo Teräs <timo.teras@iki.fi> :
> [...]
> 
> Acked-by: Francois Romieu <romieu@fr.zoreil.com>
> 
> desc->opts2 is now a bit different in the RxRes path - the only path
> where the patch can be noticed. I hope it won't uncloak something else.

Francois, do you want me to apply this directly or do you want
to submit this yourself along with any other r8169 changes you
might have queued up?

Thanks.

Re: [PATCH net] r8169: remove the obsolete and incorrect AMD workaround

[Timo Teras]

On Wed, 23 Jan 2013 01:05:41 +0100 Francois Romieu
<romieu@fr.zoreil.com> wrote:

> Timo Teräs <timo.teras@iki.fi> :
> [...]
> 
> Acked-by: Francois Romieu <romieu@fr.zoreil.com>
> 
> desc->opts2 is now a bit different in the RxRes path - the only path
> where the patch can be noticed. I hope it won't uncloak something
> else.

I was wondering what you mean by this. But as it seems
rtl8169_rx_vlan_tag() is doing opts2 = 0 in receive path. So the
only effect the workaround would have had was for the error path
packets.

So if this worries you, would it make sense to just unconditionally set
opts2 to zero also in the error path?

Actually, this sounds wrong. Why is rtl8169_rx_vlan_tag() which is
fiddling opts2 invoked *after* rtl8169_mark_to_asic() ? This means it
can overwrite the tag info if the queue is full.

- Timo

Re: [PATCH net] r8169: remove the obsolete and incorrect AMD workaround

[David Miller]

From: Francois Romieu <romieu@fr.zoreil.com>
Date: Wed, 23 Jan 2013 01:05:41 +0100

> Timo Teräs <timo.teras@iki.fi> :
> [...]
> 
> Acked-by: Francois Romieu <romieu@fr.zoreil.com>

No reply, so I am just going to assume you wanted me to integrate
this directly, so I've done so.

Re: [PATCH net] r8169: remove the obsolete and incorrect AMD workaround

[Francois Romieu]

Timo Teras <timo.teras@iki.fi> :
[...]
> Actually, this sounds wrong. Why is rtl8169_rx_vlan_tag() which is
> fiddling opts2 invoked *after* rtl8169_mark_to_asic() ? This means it
> can overwrite the tag info if the queue is full.

Yes. It has been there for ages.

What about something like the patch below ?

diff --git a/drivers/net/ethernet/realtek/r8169.c b/drivers/net/ethernet/realtek/r8169.c
index c28bc31..1170232 100644
--- a/drivers/net/ethernet/realtek/r8169.c
+++ b/drivers/net/ethernet/realtek/r8169.c
@@ -1826,8 +1826,6 @@ static void rtl8169_rx_vlan_tag(struct RxDesc *desc, struct sk_buff *skb)
 
 	if (opts2 & RxVlanTag)
 		__vlan_hwaccel_put_tag(skb, swab16(opts2 & 0xffff));
-
-	desc->opts2 = 0;
 }
 
 static int rtl8169_gset_tbi(struct net_device *dev, struct ethtool_cmd *cmd)
@@ -6064,8 +6062,6 @@ static int rtl_rx(struct net_device *dev, struct rtl8169_private *tp, u32 budget
 			    !(status & (RxRWT | RxFOVF)) &&
 			    (dev->features & NETIF_F_RXALL))
 				goto process_pkt;
-
-			rtl8169_mark_to_asic(desc, rx_buf_sz);
 		} else {
 			struct sk_buff *skb;
 			dma_addr_t addr;
@@ -6086,16 +6082,14 @@ process_pkt:
 			if (unlikely(rtl8169_fragmented_frame(status))) {
 				dev->stats.rx_dropped++;
 				dev->stats.rx_length_errors++;
-				rtl8169_mark_to_asic(desc, rx_buf_sz);
-				continue;
+				goto release_descriptor;
 			}
 
 			skb = rtl8169_try_rx_copy(tp->Rx_databuff[entry],
 						  tp, pkt_size, addr);
-			rtl8169_mark_to_asic(desc, rx_buf_sz);
 			if (!skb) {
 				dev->stats.rx_dropped++;
-				continue;
+				goto release_descriptor;
 			}
 
 			rtl8169_rx_csum(skb, status);
@@ -6111,6 +6105,10 @@ process_pkt:
 			tp->rx_stats.bytes += pkt_size;
 			u64_stats_update_end(&tp->rx_stats.syncp);
 		}
+release_descriptor:
+		desc->opts2 = 0;
+		wmb();
+		rtl8169_mark_to_asic(desc, rx_buf_sz);
 	}
 
 	count = cur_rx - tp->cur_rx;

Re: [PATCH net] r8169: remove the obsolete and incorrect AMD workaround

[Timo Teras]

On Wed, 23 Jan 2013 23:38:34 +0100 Francois Romieu
<romieu@fr.zoreil.com> wrote:

> Timo Teras <timo.teras@iki.fi> :
> [...]
> > Actually, this sounds wrong. Why is rtl8169_rx_vlan_tag() which is
> > fiddling opts2 invoked *after* rtl8169_mark_to_asic() ? This means
> > it can overwrite the tag info if the queue is full.
> 
> Yes. It has been there for ages.
> 
> What about something like the patch below ?

Looks good as the minimum intrusion for stable branches. Though, I'm
wondering if it'd be easy to refactor rtl_rx() so that it would
not need that many goto labels.