Hi all, First of all, if this does not apply cleanly somewhere, accept my apologies. The patchsets have grown organically for the last 1.5 years and I am having trouble disentangling some of them into distinct features. Feel free to bounce, I will be happy to fix it. This is taken vs 3.3.8. The choice of kernel is because this is the production Openwrt AA kernel which what we use for development. It applies with minimal fuss from 3.2 to 3.8 (I have not tested it further). Attached are the l2tpv3 and the raw transports. Principles of operation: UML original network infrastructure allocates a SKB at a time, reads into it using a driver read function and if the read is successful passes the skb to if_rx. I This will not work for multi-packet receive unless you copy. Without multipacket receive it is difficult to go beyond 1.5Gbit for L2TPv3 and 1Gbit for raw. With multipacket you are looking at double that (at least). As an alternative to multipacket rx raw can also use packet mmap. While this incurs an extra copy the savings from fewer syscalls allow it to get fairly reasonable performance. Multipacket tx is not used for now. To avoid the copy we pre-allocate a vector of skbs, a matching vector of supporting messages for recvmsg/sendmsg and matching iovs. If the encapsulation has a distinct header (l2tpv3, gre, etc) we use readmsg (or readmmsg) to split that header from the packet payload on rx. If there is no header (raw) - we read directly. Same on xmit. So the structures for the former and the latter differ somewhat - the former uses a pointer to a two buffer iov (one fo header, one for payload), the latter a pointer to a single buffer iov in the msg_hdr structures. Otherwise, they are similar. In order to avoid invoking kernel side functions directly the 3 kernel side bits we need to build these vectors are isolated in net_extra_kern.c The userspace bits including wrappers around recvmsg, sendmsg and recvmmsg are in net_extra_user.c The same file contains all the infra to build the vectors. When using vector io we no longer need drop_skb too. If there is no skb alloced (buffer length set to zero or buffer is NULL), recvmsg and recvmmsg will drop the packet for us. >>From there on, the transports use this common infra. The arguments for l2tpv3 are: eth1=l2tpv3,,src,srcport,dst,dstport,rxcookie,txcookie,rxsession,txsession,unused,mode mode is a bitmask (described in the uml_l2tpv3.h include file). 1 /* on for v6, off for v4 */ 2 /* on for udp, off for raw ip */ 4 /* cookie present */ 8 /* on for 64 bit cookie */ 16 /* draft keyed ip - no counter */ src, srcport - source, format according to the above dst, dstport - destination (optional, if null will listen for first packet) txcookie, rxcookie, txsession, rxsession - same as for ip l2tpv3 arguments unused - older mode spec, not used now mode - current (bitmask) mode spec Examples: eth1=l2tpv3,,192.168.64.1,,192.168.128.1,,0xdeadbeefdeadbeef,0xbeefdeadbeefdead,0xffffffff,0xffffffff,,c This will configure raw ip v4 EoGRE tunnel from 64.1 to 128.1. The config is not very user friendly, but it is a bit difficult to make it so using current option parsing routines. Raw is much simpler: eth1=raw,,ifname ifname is the name of the interface to bind to. Note - the driver does not bring up the interface or set its mode. For most use cases you have to do an ifconfig ethX up promisc before binding to it for this to work. A. On 28/02/14 08:33, Richard Weinberger wrote: > Am 28.02.2014 09:27, schrieb Anton Ivanov (antivano): >> Hi Richard, Hi Jeff, hi list, >> >> On behalf of Cisco systems, I am authorized to make a offer a set bug >> fixes as well as contribute several additional features and performance >> improvements to UML. All of these have been used internally for a couple >> of years and will ship as parts of product(s) in the near future. Some >> of these improve performance by up to 8 times on use cases which are of >> interest to us and are likely to be of interest to the community. >> >> As the full patchset is now in the 100k+ zone, so I am going to do only >> the announcement now and submit the patches one by one after that over >> the next 1-2 weeks. >> >> We will submit separately bug fixes for: >> >> 1. Critical memory corruption on startup observed on heavily loaded >> machines (especially when multiple UMLs run simultaneously). >> 2. Fix(es) for incorrect handling of error conditions when UML is run >> under expect and conX=fd: is used to communicate with another process. >> The same error may be observed on internal UML IPCs too leading to >> immediate crash. >> >> I will also file bugs for both vs Debian UML package so that patches for >> both can go in ASAP. >> >> In addition to the bug fixes, the new features include: >> >> 1. Several transports. All can do up to multi-gigabit throughput on some >> scenarios. We are contributing their counterparts to qemu/kvm as well. >> >> 1.1. Direct connection of UML to overlay networks/L2 VPNs using L2TPv3. >> >> This has a number of advantages compared to the existing UML "multicast" >> and qemu "socket" transports. >> >> * Standard compliant - RFC 3931 updated recently by RFC 5641 >> * Supported on most network equipment >> * Allowing to move virtual switching off-host to an NPU or high >> performance physical switch >> * Allowing to mix virtual and physical switching (well supported on >> modern Linuxes and other OSes) >> * Well researched security profile as well as established >> interactions with IPSEC allowing to extend virtual networks outside the >> datacenter to remote physical devices and/or VMs. >> >> 1.2. Raw transport which allows both bi-directional communication with >> any network device which looks like Ethernet as well as in-span >> listening at speeds in the multi-gigabit range. >> >> 1.3. We intend to contribute other key overlay transports like GRE, etc >> as well. The ones we are contributing at this point are the ones which >> we have used most extensively and have had the most testing (~ 1.5-2 years). >> >> 2. New high res timer subsystem >> >> Adding these new network transports to UML revealed a key issue - it >> cannot meter or shape any traffic correctly as its internal timer system >> is way off. Personally, I consider it a bug, however there is no "easy" >> fix here. The only way to fix it is a new timer driver. Unfortunately, >> it does not fix uml userspace - timers there remain off. It does fix all >> kernel timer functionality - traffic shaping (both qdisc and iptables >> traffic limits). >> >> As a side effect, this provides performance improvements for tcp and >> other protocols which rely on kernel high res timers for their state >> machines. >> >> We have further scalability contributions lined up which improve network >> and IO performance between 1.5 and 8 times (depending on use case), >> allow hundreds of virtual interfaces per UML without performance >> penalties, allow to run several hundreds (if not thousands) of UMLs per >> machine, etc. All in all, it can no go where no virtualization and no >> virtual networking has gone before. >> >> However, I would prefer to take it one step at a time and get through >> these first (even these are quite a lot for one "sitting"). > Sounds awesome! > > Please send the patches as soon as possible. > I'm eager to test and merge them. > > Thanks, > //richard > > ------------------------------------------------------------------------------ > Flow-based real-time traffic analytics software. Cisco certified tool. > Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer > Customize your own dashboards, set traffic alerts and generate reports. > Network behavioral analysis & security monitoring. All-in-one tool. > http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.clktrk > _______________________________________________ > User-mode-linux-devel mailing list > User-mode-linux-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel >