Re: VLAN egress performance

[Patrick McHardy <kaber@trash.net>]

Mattias Rönnblom wrote:
> Hi.
> 
> I'm running Linux on PC w/ a Core i7 CPU and two Intel 82598 NICs, and
> I see some anomalies when it comes to egress VLAN performance. I
> thought maybe someone on this list was interested in my results.
> 
> I'm running the stock Ubuntu 2.6.31 kernel, but with a newer ixgbe
> driver (2.0.44.14).
> 
> The benchmark is IP forwarding with unidirectional UDP flows @ 64 byte
> packets, and I get:
> 
> Ingress VLAN  Egress VLAN  Packet Rate    CPU utilization (all cores)
> No            No           5.0 Mpacket/s  ~70%
> Yes           No           5.0 Mpacket/s  ~75%
> No            Yes          1.4 Mpacket/s  ~26%
> Yes           Yes          1.3 Mpacket/s  ~26%
> 
> "VLAN" here mean I've put a VLAN device on top of the real ixgbe
> device.
> 
> As you can see, if the egress i/f is a VLAN i/f, the performance is
> reduced to less than a third. And in the case of egress VLAN, the
> systems basically only uses one HW thread (with a softirqd process
> taking up all the time).
> 
> Enabling lockdep, it looks like execution is serialized to a large
> extent by contention around the "vlan_netdev_xmit_lock_key" lock.
> 
> I call this anomaly because I was surprised to see it (as oppose to
> other performance degradations/scalability issues in the area of
> multicore and IP traffic handling performance).

2.6.32 contains VLAN multiqueue support and should scale better.