netmap, VALE and netmap pipes

netmap, VALE and netmap pipes

[Luigi Rizzo]

Hi,
we have recently made a few extensions to netmap/VALE and put various
pieces of code on public repositories, so i thought i'd share the
pointers. All the code below runs with equal features and performance
on FreeBSD and Linux, and we are trying to upstream it in the relevant
projects if possible (as an example, QEMU recently added a netmap backend),
at which point some of these clone repositories will become unnecessary.

See http://info.iet.unipi.it/~luigi/netmap for more details.

https://code.google.com/p/netmap/
    The latest netmap code for FreeBSD/Linux. It has native support
    for certain NICs; emulated netmap over unmodified drivers;
    enhanced parallelism in the VALE switch (20 Mpps/source, scaling
    up to ~50Mpps); and a new feature called "netmap pipe" that
    does zero-copy blocking I/O at over 100 Mpps.
        Other features are the ability to allocate tons of extra
    netmap buffers, and configurable sharing of memory among NICs,
    VALE ports and netmap pipes. This increases the opportunity for
    zero copy operation.
        The user API is also greatly simplified, with a naming
    scheme that permits easy access to all types of ports including
    individual NIC queues.

https://code.google.com/p/netmap-libpcap
    a netmap-enabled version of libpcap. With this, basically any
    pcap client can read/write traffic at 10+ Mpps, with zerocopy
    reads and (soon) support for zerocopy writes. Whether applications
    can cope with these packet rates, of course, is another story.

https://code.google.com/p/netmap-click
    a netmap-enabled version of the Click Modular Router.  This
    code matches the current version of netmap, supporting all
    features (including netmap pipes).

https://code.google.com/p/netmap-ipfw
    a netmap-enabled, userspace version of the ipfw firewall and
    dummynet network emulator. This version reaches 7-10 Mpps for
    filtering and over 2.5 Mpps for emulation.


Hope you'll find it useful.

cheers
luigi
_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"

Re: netmap, VALE and netmap pipes

[Kevin Bowling]

On 2/17/2014 3:11 AM, Luigi Rizzo wrote:
> Hi,
> we have recently made a few extensions to netmap/VALE and put various
> pieces of code on public repositories, so i thought i'd share the
> pointers. All the code below runs with equal features and performance
> on FreeBSD and Linux, and we are trying to upstream it in the relevant
> projects if possible (as an example, QEMU recently added a netmap backend),
> at which point some of these clone repositories will become unnecessary.

Just a thought, maybe this is a good time for The FreeBSD Foundation to 
reach out to The Linux Foundation for lobbying netmap into their main 
line kernel.  It would be nice if netmap becomes the de facto UNIX 
standard for this type of programming (it is vendor neutral and broadly 
applicable vs other solutions), and avoid not-invented-here APIs like 
non-blocking I/O went through with all the UNIX flavors.

Regards,
Kevin Bowling

_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"

Re: netmap, VALE and netmap pipes

[Stephen Hemminger]

On Mon, 17 Feb 2014 12:12:36 -0700
Kevin Bowling <kevin.bowling@kev009.com> wrote:

> On 2/17/2014 3:11 AM, Luigi Rizzo wrote:
> > Hi,
> > we have recently made a few extensions to netmap/VALE and put various
> > pieces of code on public repositories, so i thought i'd share the
> > pointers. All the code below runs with equal features and performance
> > on FreeBSD and Linux, and we are trying to upstream it in the relevant
> > projects if possible (as an example, QEMU recently added a netmap backend),
> > at which point some of these clone repositories will become unnecessary.
> 
> Just a thought, maybe this is a good time for The FreeBSD Foundation to 
> reach out to The Linux Foundation for lobbying netmap into their main 
> line kernel.  It would be nice if netmap becomes the de facto UNIX 
> standard for this type of programming (it is vendor neutral and broadly 
> applicable vs other solutions), and avoid not-invented-here APIs like 
> non-blocking I/O went through with all the UNIX flavors.
> 
> Regards,
> Kevin Bowling

You do not understand the role of Linux Foundation.
Lobbying would only serve to annoy the developers.

Netmap was submitted and rejected for a number of issues.
Read the netdev mailing list archives if you want to follow what
is going on.

Re: netmap, VALE and netmap pipes

[Kevin Bowling]

On 2/17/2014 1:14 PM, Stephen Hemminger wrote:
> On Mon, 17 Feb 2014 12:12:36 -0700
> Kevin Bowling <kevin.bowling@kev009.com> wrote:
>
>> On 2/17/2014 3:11 AM, Luigi Rizzo wrote:
>>> Hi,
>>> we have recently made a few extensions to netmap/VALE and put various
>>> pieces of code on public repositories, so i thought i'd share the
>>> pointers. All the code below runs with equal features and performance
>>> on FreeBSD and Linux, and we are trying to upstream it in the relevant
>>> projects if possible (as an example, QEMU recently added a netmap backend),
>>> at which point some of these clone repositories will become unnecessary.
>>
>> Just a thought, maybe this is a good time for The FreeBSD Foundation to
>> reach out to The Linux Foundation for lobbying netmap into their main
>> line kernel.  It would be nice if netmap becomes the de facto UNIX
>> standard for this type of programming (it is vendor neutral and broadly
>> applicable vs other solutions), and avoid not-invented-here APIs like
>> non-blocking I/O went through with all the UNIX flavors.
>>
>> Regards,
>> Kevin Bowling

Hi Stephen,

> You do not understand the role of Linux Foundation.
> Lobbying would only serve to annoy the developers.

I attempted to say that with tongue in cheek and am sorry if I offended 
anyone.

> Netmap was submitted and rejected for a number of issues.
> Read the netdev mailing list archives if you want to follow what
> is going on.

I am not familiar with the situation on the Linux side.  My hope is that 
the opportunity window is not lost if there is a possibility for one API 
on both platforms.

Regards,
Kevin Bowling

Re: netmap, VALE and netmap pipes

[jerry]

Hi Luigi,

How to use netmap pipe by pkt-gen commands?
I have tried the commands as follows:
  ./pkt-gen -i netmap:pipename{1 -f tx
  ./pkt-gen -i netmap:pipename}1 -f rx  (in another terminal)
But it works failed.
Should the pipename be replaced with a invalid NIC name such as "eth3" in netmap mode?

The netmap pipe works from software ring to software ring independently with NICs, I understand. Is that right?

B.R.
Jerry

On 2014/2/17 18:11, Luigi Rizzo wrote:
> Hi,
> we have recently made a few extensions to netmap/VALE and put various
> pieces of code on public repositories, so i thought i'd share the
> pointers. All the code below runs with equal features and performance
> on FreeBSD and Linux, and we are trying to upstream it in the relevant
> projects if possible (as an example, QEMU recently added a netmap backend),
> at which point some of these clone repositories will become unnecessary.
> 
> See http://info.iet.unipi.it/~luigi/netmap for more details.
> 
> https://code.google.com/p/netmap/
>     The latest netmap code for FreeBSD/Linux. It has native support
>     for certain NICs; emulated netmap over unmodified drivers;
>     enhanced parallelism in the VALE switch (20 Mpps/source, scaling
>     up to ~50Mpps); and a new feature called "netmap pipe" that
>     does zero-copy blocking I/O at over 100 Mpps.
>         Other features are the ability to allocate tons of extra
>     netmap buffers, and configurable sharing of memory among NICs,
>     VALE ports and netmap pipes. This increases the opportunity for
>     zero copy operation.

Re: netmap, VALE and netmap pipes

[Luigi Rizzo]

On Fri, Feb 21, 2014 at 5:03 PM, jerry <jerry.lilijun@huawei.com> wrote:

> Hi Luigi,
>
> How to use netmap pipe by pkt-gen commands?
> I have tried the commands as follows:
>   ./pkt-gen -i netmap:pipename{1 -f tx
>   ./pkt-gen -i netmap:pipename}1 -f rx  (in another terminal)
> But it works failed.
> Should the pipename be replaced with a invalid NIC name such as "eth3" in
> netmap mode?
>

netmap: expects the name of an existing device.
Arbitrary names should have the 'vale' prefix e.g.

./pkt-gen -i valexx:p{0 -f tx
./pkt-gen -i valexx:p}0 -f rx



>
> The netmap pipe works from software ring to software ring independently
> with NICs, I understand. Is that right?
>
>
correct, but the basename is used to group ports
(NICs/VALE ports and netmap pipes) that use the same
shared memory so that you can do zero copy among them.

cheers
luigi
 

> B.R.
> Jerry
>
> On 2014/2/17 18:11, Luigi Rizzo wrote:
> > Hi,
> > we have recently made a few extensions to netmap/VALE and put various
> > pieces of code on public repositories, so i thought i'd share the
> > pointers. All the code below runs with equal features and performance
> > on FreeBSD and Linux, and we are trying to upstream it in the relevant
> > projects if possible (as an example, QEMU recently added a netmap
> backend),
> > at which point some of these clone repositories will become unnecessary.
> >
> > See http://info.iet.unipi.it/~luigi/netmap for more details.
> >
> > https://code.google.com/p/netmap/
> >     The latest netmap code for FreeBSD/Linux. It has native support
> >     for certain NICs; emulated netmap over unmodified drivers;
> >     enhanced parallelism in the VALE switch (20 Mpps/source, scaling
> >     up to ~50Mpps); and a new feature called "netmap pipe" that
> >     does zero-copy blocking I/O at over 100 Mpps.
> >         Other features are the ability to allocate tons of extra
> >     netmap buffers, and configurable sharing of memory among NICs,
> >     VALE ports and netmap pipes. This increases the opportunity for
> >     zero copy operation.
>
>
>
>
>


-- 
-----------------------------------------+-------------------------------
 Prof. Luigi RIZZO, rizzo@iet.unipi.it  . Dip. di Ing. dell'Informazione
 http://www.iet.unipi.it/~luigi/        . Universita` di Pisa
 TEL      +39-050-2211611               . via Diotisalvi 2
 Mobile   +39-338-6809875               . 56122 PISA (Italy)
-----------------------------------------+-------------------------------
_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"

netmap, VALE and netmap pipes

[Luigi Rizzo]

Hi,
we have recently made a few extensions to netmap/VALE and put various
pieces of code on public repositories, so i thought i'd share the
pointers. All the code below runs with equal features and performance
on FreeBSD and Linux, and we are trying to upstream it in the relevant
projects if possible (as an example, QEMU recently added a netmap backend),
at which point some of these clone repositories will become unnecessary.

See http://info.iet.unipi.it/~luigi/netmap for more details.

https://code.google.com/p/netmap/
    The latest netmap code for FreeBSD/Linux. It has native support
    for certain NICs; emulated netmap over unmodified drivers;
    enhanced parallelism in the VALE switch (20 Mpps/source, scaling
    up to ~50Mpps); and a new feature called "netmap pipe" that
    does zero-copy blocking I/O at over 100 Mpps.
        Other features are the ability to allocate tons of extra
    netmap buffers, and configurable sharing of memory among NICs,
    VALE ports and netmap pipes. This increases the opportunity for
    zero copy operation.
        The user API is also greatly simplified, with a naming
    scheme that permits easy access to all types of ports including
    individual NIC queues.

https://code.google.com/p/netmap-libpcap
    a netmap-enabled version of libpcap. With this, basically any
    pcap client can read/write traffic at 10+ Mpps, with zerocopy
    reads and (soon) support for zerocopy writes. Whether applications
    can cope with these packet rates, of course, is another story.

https://code.google.com/p/netmap-click
    a netmap-enabled version of the Click Modular Router.  This
    code matches the current version of netmap, supporting all
    features (including netmap pipes).

https://code.google.com/p/netmap-ipfw
    a netmap-enabled, userspace version of the ipfw firewall and
    dummynet network emulator. This version reaches 7-10 Mpps for
    filtering and over 2.5 Mpps for emulation.


Hope you'll find it useful.

cheers
luigi