Re: eth_type_trans(): Re: [Bug #11308] tbench regression on each kernel release from 2.6.22 -> 2.6.28

[Eric Dumazet <dada1@cosmosbay.com>]

[-- Attachment #1: Type: text/plain, Size: 1648 bytes --]

Eric Dumazet a écrit :
> 
> But seeing your disassembly, I can see compare_ether_addr() is not inlined.
> 
> This sucks.
> 
> /**
> * compare_ether_addr - Compare two Ethernet addresses
> * @addr1: Pointer to a six-byte array containing the Ethernet address
> * @addr2: Pointer other six-byte array containing the Ethernet address
> *
> * Compare two ethernet addresses, returns 0 if equal
> */
> static inline unsigned compare_ether_addr(const u8 *addr1, const u8 *addr2)
> {
>        const u16 *a = (const u16 *) addr1;
>        const u16 *b = (const u16 *) addr2;
> 
>        BUILD_BUG_ON(ETH_ALEN != 6);
>        return ((a[0] ^ b[0]) | (a[1] ^ b[1]) | (a[2] ^ b[2])) != 0;
> }
> 
> On my machine/compiler, it is inlined, that makes a big difference.

old gcc compiler... OK understood...

> 
> c0420750 <eth_type_trans>: /* eth_type_trans total:  14417  0.4101 */
> 
> 

Could you try this patch Ingo ?

Thanks

[PATCH] net: eth_type_trans() should be a leaf function

In old days, eth_type_trans() was a leaf function. It is not anymore the case.

eth_type_trans() is a critical network function, called for each incoming packet.

We should make sure it is not calling functions, especially trivial ones.

1) Adds an __always_inline to compare_ether_addr() : This one was created to be faster
   than memcmp(). It really should be faster (and inlined)

2) Hand code skb_put() call in eth_type_trans()

Signed-off-by: Eric Dumazet <dada1@cosmosbay.com>
---
 include/linux/etherdevice.h |    2 +-
 net/ethernet/eth.c          |    7 ++++++-
 2 files changed, 7 insertions(+), 2 deletions(-)

[-- Attachment #2: eth_type_trans_speedup.patch --]
[-- Type: text/plain, Size: 1053 bytes --]

diff --git a/include/linux/etherdevice.h b/include/linux/etherdevice.h
index 25d62e6..94af6a7 100644
--- a/include/linux/etherdevice.h
+++ b/include/linux/etherdevice.h
@@ -128,7 +128,7 @@ static inline void random_ether_addr(u8 *addr)
  *
  * Compare two ethernet addresses, returns 0 if equal
  */
-static inline unsigned compare_ether_addr(const u8 *addr1, const u8 *addr2)
+static __always_inline unsigned compare_ether_addr(const u8 *addr1, const u8 *addr2)
 {
 	const u16 *a = (const u16 *) addr1;
 	const u16 *b = (const u16 *) addr2;
diff --git a/net/ethernet/eth.c b/net/ethernet/eth.c
index b9d85af..30b60b2 100644
--- a/net/ethernet/eth.c
+++ b/net/ethernet/eth.c
@@ -162,7 +162,12 @@ __be16 eth_type_trans(struct sk_buff *skb, struct net_device *dev)
 
 	skb->dev = dev;
 	skb_reset_mac_header(skb);
-	skb_pull(skb, ETH_HLEN);
+	/*
+	 * Hand coded skb_pull(skb, ETH_HLEN) to avoid a function call
+	 */
+	if (likely(skb->len >= ETH_HLEN))
+		__skb_pull(skb, ETH_HLEN);
+
 	eth = eth_hdr(skb);
 
 	if (is_multicast_ether_addr(eth->h_dest)) {