[RFC] skb align patch

[RFC] skb align patch

[Stephen Hemminger]

Based on the Intel suggestion that PCI-express overhead is
a significant cost.

Would people doing performance please measure the impact of
changing SKB alignment (64 bit only).


--- a/arch/x86/include/asm/system.h	2009-09-20 14:08:40.922346912 -0700
+++ b/arch/x86/include/asm/system.h	2009-09-20 14:14:37.012371200 -0700
@@ -455,4 +455,14 @@ static inline void rdtsc_barrier(void)
 	alternative(ASM_NOP3, "lfence", X86_FEATURE_LFENCE_RDTSC);
 }
 
+#ifndef CONFIG_X86_32
+/*
+ * DMA to unaligned address is more expensive than the the
+ * overhead of unaligned CPU access.
+ */
+#define NET_IP_ALIGN	0
+#define NET_SKB_PAD	L1_CACHE_BYTES
+#endif
+
+
 #endif /* _ASM_X86_SYSTEM_H */

Re: [RFC] skb align patch

[Eric Dumazet]

Stephen Hemminger a écrit :
> Based on the Intel suggestion that PCI-express overhead is
> a significant cost.
> 
> Would people doing performance please measure the impact of
> changing SKB alignment (64 bit only).

I had this idea some time ago when I hit a limit on bnx2 adapter
(Giga bit link, BCM5708S), with small packets. pktgen was able
to send ~500 Mbps 'only', or 700kps if I remember well.
So I tried to align the pktgen build packet to a cache line,
it gave no difference at all, but it was on a 32 bit kernel.
(Thus my patch was for pktgen only, not a generic one as yours)

Could you elaborate why this change could be useful on 64bit ?

Thanks

> 
> 
> --- a/arch/x86/include/asm/system.h	2009-09-20 14:08:40.922346912 -0700
> +++ b/arch/x86/include/asm/system.h	2009-09-20 14:14:37.012371200 -0700
> @@ -455,4 +455,14 @@ static inline void rdtsc_barrier(void)
>  	alternative(ASM_NOP3, "lfence", X86_FEATURE_LFENCE_RDTSC);
>  }
>  
> +#ifndef CONFIG_X86_32
> +/*
> + * DMA to unaligned address is more expensive than the the
> + * overhead of unaligned CPU access.
> + */
> +#define NET_IP_ALIGN	0
> +#define NET_SKB_PAD	L1_CACHE_BYTES
> +#endif
> +
> +
>  #endif /* _ASM_X86_SYSTEM_H */
> --

Re: [RFC] skb align patch

[Eric Dumazet]

Stephen Hemminger a écrit :
> On Mon, 21 Sep 2009 08:13:20 +0200
> Eric Dumazet <eric.dumazet@gmail.com> wrote:
> 
>> Stephen Hemminger a écrit :
>>> Based on the Intel suggestion that PCI-express overhead is
>>> a significant cost.
>>>
>>> Would people doing performance please measure the impact of
>>> changing SKB alignment (64 bit only).
>> I had this idea some time ago when I hit a limit on bnx2 adapter
>> (Giga bit link, BCM5708S), with small packets. pktgen was able
>> to send ~500 Mbps 'only', or 700kps if I remember well.
>> So I tried to align the pktgen build packet to a cache line,
>> it gave no difference at all, but it was on a 32 bit kernel.
>> (Thus my patch was for pktgen only, not a generic one as yours)
>>
>> Could you elaborate why this change could be useful on 64bit ?
>>
> 
> It is useful on all architecture where unaligned CPU access is
> relatively cheap.
> 
> The issue is that a unaligned DMA requires a read/modify/write
> cache line access versus just a write access. I am not a bus
> expert, but writes are probably more pipelined as well.
> 

Oh I see, you want to optimize the rx (NIC has to do a DMA
to write packet into host memory and this DMA could be a read
/modify/write if address is not aligned, instead of a pure write),
 while I tried to align skb to optimize the pktgen tx 
(NIC has to do a DMA to  read packet from host), and align the skb
had no effect.

Maybe we should separate the rx/tx, and try your idea only
for skb allocated for rx.

Also/Or we might try 
__builtin_prefetch (addr, 0, 0);
to instruct cpu to commit to memory cache lines that are
going to be modified by NIC.

Re: [RFC] skb align patch

[Stephen Hemminger]

On Mon, 21 Sep 2009 08:13:20 +0200
Eric Dumazet <eric.dumazet@gmail.com> wrote:

> Stephen Hemminger a écrit :
> > Based on the Intel suggestion that PCI-express overhead is
> > a significant cost.
> > 
> > Would people doing performance please measure the impact of
> > changing SKB alignment (64 bit only).
> 
> I had this idea some time ago when I hit a limit on bnx2 adapter
> (Giga bit link, BCM5708S), with small packets. pktgen was able
> to send ~500 Mbps 'only', or 700kps if I remember well.
> So I tried to align the pktgen build packet to a cache line,
> it gave no difference at all, but it was on a 32 bit kernel.
> (Thus my patch was for pktgen only, not a generic one as yours)
> 
> Could you elaborate why this change could be useful on 64bit ?
> 

It is useful on all architecture where unaligned CPU access is
relatively cheap.

The issue is that a unaligned DMA requires a read/modify/write
cache line access versus just a write access. I am not a bus
expert, but writes are probably more pipelined as well.

-- 

Re: [RFC] skb align patch

[Stephen Hemminger]

On Tue, 22 Sep 2009 05:20:53 +0200
Eric Dumazet <eric.dumazet@gmail.com> wrote:

> Stephen Hemminger a écrit :
> > On Mon, 21 Sep 2009 08:13:20 +0200
> > Eric Dumazet <eric.dumazet@gmail.com> wrote:
> > 
> >> Stephen Hemminger a écrit :
> >>> Based on the Intel suggestion that PCI-express overhead is
> >>> a significant cost.
> >>>
> >>> Would people doing performance please measure the impact of
> >>> changing SKB alignment (64 bit only).
> >> I had this idea some time ago when I hit a limit on bnx2 adapter
> >> (Giga bit link, BCM5708S), with small packets. pktgen was able
> >> to send ~500 Mbps 'only', or 700kps if I remember well.
> >> So I tried to align the pktgen build packet to a cache line,
> >> it gave no difference at all, but it was on a 32 bit kernel.
> >> (Thus my patch was for pktgen only, not a generic one as yours)
> >>
> >> Could you elaborate why this change could be useful on 64bit ?
> >>
> > 
> > It is useful on all architecture where unaligned CPU access is
> > relatively cheap.
> > 
> > The issue is that a unaligned DMA requires a read/modify/write
> > cache line access versus just a write access. I am not a bus
> > expert, but writes are probably more pipelined as well.
> > 
> 
> Oh I see, you want to optimize the rx (NIC has to do a DMA
> to write packet into host memory and this DMA could be a read
> /modify/write if address is not aligned, instead of a pure write),
>  while I tried to align skb to optimize the pktgen tx 
> (NIC has to do a DMA to  read packet from host), and align the skb
> had no effect.
> 
> Maybe we should separate the rx/tx, and try your idea only
> for skb allocated for rx.
> 
> Also/Or we might try 
> __builtin_prefetch (addr, 0, 0);
> to instruct cpu to commit to memory cache lines that are
> going to be modified by NIC.

Don't think it matters whether RX buffer has to read/modify/write
from cpu cache or memory on modern cache snooping architecures.
The cost is the PCI traffic.

-- 

Re: [RFC] skb align patch

[David Miller]

From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Tue, 22 Sep 2009 05:20:53 +0200

> Oh I see, you want to optimize the rx (NIC has to do a DMA to write
> packet into host memory and this DMA could be a read /modify/write
> if address is not aligned, instead of a pure write), while I tried
> to align skb to optimize the pktgen tx (NIC has to do a DMA to read
> packet from host), and align the skb had no effect.

This is a problem with these kinds of changes.

This patch from Stephen came out of a presentation and discussion
at netconf where the Intel folks showed that if they did a combination
of things it improved NUMA forwarding numbers a lot.

So you couldn't just do NUMA spreading of RX queue memory, or just
do this ALIGN patch, or just eliminate the false sharing from
statistics updates.

You had to do all three to start seeing forwarding rates go up.

So don't worry, this is getting us somewhere to where improvement
shows, but individually each change won't trigger it.

The alignment in this patch is a real big deal for 64 byte forwarding
tests, where the entire packet is a whole PCI-E cacheline.  But not
if it isn't aligned properly.

Re: [RFC] skb align patch

[Thomas Graf]

On Mon, Sep 21, 2009 at 10:29:40PM -0700, David Miller wrote:
> The alignment in this patch is a real big deal for 64 byte forwarding
> tests, where the entire packet is a whole PCI-E cacheline.  But not
> if it isn't aligned properly.

As I pointed out to Herbert already, this alignment change may actually
make things worse or even break things as long as compare_ether_header()
used in __napi_gro_receive() expects the IP header to be aligned to 4
bytes. That can be fixed of course, just wanted to mention it.

Re: [RFC] skb align patch

[Herbert Xu]

Thomas Graf <tgraf@infradead.org> wrote:
>
> As I pointed out to Herbert already, this alignment change may actually
> make things worse or even break things as long as compare_ether_header()
> used in __napi_gro_receive() expects the IP header to be aligned to 4
> bytes. That can be fixed of course, just wanted to mention it.

Well this particular patch doesn't because x86-64 is fine with
unaligned access :)

But yeah we need to fix up GRO to not do that for other arches.
It's on my list.

Thanks,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt