Re: SKY2 vs SK98LIN performance on 88E8053 MAC

[Stephen Hemminger <shemminger@linux-foundation.org>]

On Fri, 8 Jun 2007 18:20:02 -0700 (PDT)
Philip Romanov <philip_romanov@yahoo.com> wrote:

> Hi, Stephen
> 
> We are doing pure IPv4 forwarding between two Ethernet
> interfaces:
> 
>  IXIA port A<--->System Under Test<--->IXIA Port B
> 
> Traffic has two IP destinations for each direction and
> L4 protocol is UDP. There are two static ARP entries
> and only interface routes. Two tests are identical
> except that we switch from one driver to another. 
> 
> Ethernet ports on the SUT are oversubscribed -- I'm
> sending 60% of line rate (of 256-byte packets) and
> measuring percentage of pass-through traffic which
> makes to the IXIA port on the other side. Traffic is
> bidirectional and system load is close to 100%.
> 
> I attach vmlinux and driver profiles I have taken with
> oprofile 0.8.2. I can easily take more
> measurements/experiemnts if need be. 
> 
> Regards,
> 
>    Philip
> 
> 
> > > We are observing severe IPv4 forwarding
> > degradation 
> > > when switching from sk98lin to sky2 driver. Setup:
> > > plain 2.6.21.3 kernel, 88E8053 Marvell Yukon2 MAC,
> > > sk98lin is @revision 8.41.2.3 coming from FC6,
> > SKY2
> > > driver from 2.6.21.3 kernel, both drivers are in
> > NAPI
> > > mode. 
> > >  
> > > Benchmarks are done using bidirectional traffic 
> > > generated by IXIA, sending 256-byte packets.
> > Observed 
> > > packet throughput is almost 30% higher with
> > sklin98
> > > driver. 
> > >  
> > > Ethernet flow control is turned off in SKY2 driver
> > 
> > > (hard-coded as off, we know about this problem).  
> > >  
> > > I also have oprofile records of the drivers in
> > case 
> > > anybody is interested.
> > >  
> > > Please share info if you know anything on SKY2
> > > performance bottlenecks.
> > >  
> > 
> > I'm surprised? The vendor driver has bogus extra
> > locking and other
> > crap. Please send profile data.  Flow control should
> > work on sky2 (now).
> > 
> > Are you routing or doing real TCP transfers?
> > 
> 

BTW Please don't top post it makes reading this confusing.

Could you post the profiles. Hopefully, others have good ideas
as well.

256 bytes is the size where the copybreak optimization kicks in
so you might want to experiment with the copybreak module option
to the sky2 driver. copybreak=0 would no packets to be copied,
copybreak=1514 would cause all packets to be copied.  Copying is
an optimization that helps when receiving small packets locally,
but may slow down forwarding path.


-- 
Stephen Hemminger <shemminger@linux-foundation.org>