Re: VLAN egress performance

Patrick McHardy
2010-02-08T13:13:28Z

Mattias Rönnblom wrote:
> Hi.
>
> I'm running Linux on PC w/ a Core i7 CPU and two Intel 82598 NICs, and
> I see some anomalies when it comes to egress VLAN performance. I
> thought maybe someone on this list was interested in my results.
>
> I'm running the stock Ubuntu 2.6.31 kernel, but with a newer ixgbe
> driver (2.0.44.14).
>
> The benchmark is IP forwarding with unidirectional UDP flows @ 64 byte
> packets, and I get:
>
> Ingress VLAN Egress VLAN Packet Rate CPU utilization (all cores)
> No No 5.0 Mpacket/s ~70%
> Yes No 5.0 Mpacket/s ~75%
> No Yes 1.4 Mpacket/s ~26%
> Yes Yes 1.3 Mpacket/s ~26%
>
> "VLAN" here mean I've put a VLAN device on top of the real ixgbe
> device.
>
> As you can see, if the egress i/f is a VLAN i/f, the performance is
> reduced to less than a third. And in the case of egress VLAN, the
> systems basically only uses one HW thread (with a softirqd process
> taking up all the time).
>
> Enabling lockdep, it looks like execution is serialized to a large
> extent by contention around the "vlan_netdev_xmit_lock_key" lock.
>
> I call this anomaly because I was surprised to see it (as oppose to
> other performance degradations/scalability issues in the area of
> multicore and IP traffic handling performance).

2.6.32 contains VLAN multiqueue support and should scale better.