Re: [PATCH v4 net-next 0/4] net: low latency Ethernet device polling

Alex Rosenbaum
2013-05-23T11:02:27Z

Eliezer,

With AmirV's help we got this working on our NIC as well and it look
nice. We too see the nice performance gain.

I tested with epoll and as expected there is no performance improvement.
I don't think there is any point delaying this feature commit due to
this fact. Future development should handle that.

I also tested LLS with different message rates.
Using sockperf you can set a ping send rate (--msp) and measure latency
at different rates (I don't think netperf can do this).
In the financial trading sector, low latencies for 100mps are just as
important as in 50Kmsp (or higher). The market orders go out at these
low rates.
I noticed a penalty in the latency performance as I go lower in mps. I
don't think it is related to the LLS code but it is more obvious then
without it since you reach lower results.

These numbers are for sockperf TCP ping-pong at different msp for a 12
byte payload.
I verified LLS hit counter was at 100% for all different message rates
on both server and client side.
Machine is a x86_64 Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz with 64GB ram.

rate | LLS-on RTT | LLS-off RTT
(msp) | (usec) | (usec)
10000 | 14.0 | 21.8
1000 | 15.6 | 23.0
100 | 16.6 | 24.4

You can see that as I go lower in send message rate the latency increases.

* Don't consider these number as best results, they or on a random
machine with some effort tuning and core isolation. I saw this hit in
performance as lower the msg rate on several machine elsewhere and I am
sure it will reproduce on your tuned machine so you can notice it as well.

Again, this should not block your feature commit but is interesting for
me to understand and I though someone here might have a good explanation.

thanks,
Alex Rosenbaum
Director R&D Application Acceleration
Mellanox Technologies | Raanana, Israel | +972 (74) 712-9215