Re: [RFC 1/2] netmap: infrastructure (in staging)

[Jamal Hadi Salim <jhs@mojatatu.com>]

On 13-04-20 07:31 AM, Daniel Borkmann wrote:

> Also, I just looked over Netmap's Usenix paper from 2012, where they
> compare netmap against pktgen, and while they state the version of the FreeBSD
> kernel
> where they did the evaluation on, they just don't even mention the Linux'
> kernel version, their Linux kernel setup etc. Not even mentioning a
> comparison
> of PF_PACKET+fanout (similarly as the PF_RING project seems to avoid this
> comparison and only presents perf numbers where they just count packets !).
> Also, I've seen other papers published in 2012 on this topic, where they
> compare performance with a 2.6.2x kernel, hm, quite sad actually.

I hope I can put your doubts to rest. Netmap does provide the 
performance it claims to. I did play with it about 6-9 months back and i 
was able to loopback wirerate 10Gbps (~14.4Mpps) 64B packets on a 
_single core_. i.e i send to from machine A to B which echoes back to 
the sender via a driver hack i had on the intel driver and i count the 
packets. I should note that this was with machines that have circa 2010 
capabilities (and they were cheap too).

It is true without some changes to the kernel (such as using multiqueues 
and batching, pktgen will not be able to achieve that speed). It would 
be interesting to see what you can achieve with PF_PACKET transmit. 
PF_PACKET is already behind if you have to depend on fanout for receive 
(you are using more processing).
Granted that this was a simple app. Unfortunately the trend with 
approaches like netmap is happening. Theres some closed source thing out 
of intel called DPDK which intel is aggressively marketing.
If someone is showing you 10x improvement over what Linux gives you,
then we are doing something wrong and we shouldnt be living in some
parallel universe and claim theres nothing to see here.
1.5-3x is something i can live with because it shows theres some room
for tweaking.

 From a personal perspective - I have always been a supporter of "if 
something is wrong with what linux gives you, lets improve it" (Still 
grinding my teeth at openvswitch).
How about we learn something from this and try to improve what we have?
I did talk to Luigi briefly (CCing him) - his biggest beef was with skbs
and how fat they are. I know Eric D. has been doing some excellent work
putting them on some low-carb diet but there are still people showing up
and arguing for more fields. Here's a thought:
Could we put something in the kernel that allows for high-perfomance 
zero copy to user  space and inject packets + metadata (from/to 
arbitrary parts of the kernel)? I do this all the time from say 
ingress/egress via tuntap (which sucks at this).
Yes, theres a danger of allowing for competing interfaces in userspace
to develop tcp stacks etc but I think for someone who wants to take 
advantage of Linux, thats a non-starter.

cheers,
jamal