Userspace network stack and netchannels.

Userspace network stack and netchannels.

[Evgeniy Polyakov]

[-- Attachment #1: Type: text/plain, Size: 1608 bytes --]

Hi.

I've released new version of the extremely small 
userspace netowork stack [1] implementation.
Stack supports TCP and UDP over IP.
It works on top of netchannels [2] or packet socket 
(returned back in this release).

Supported features:
	* TCP/UDP sending and receiving.
	* Timestamp, window scaling, MSS TCP options.
	* PAWS.
	* Slow start and congestion control.
	* Socket-like interface.
	* Retransmit and fast retransmit support.
	* support for TCP listen state (only point-to-point mode,
		i.e. no new data channels are created, when new client is
		connected, instead state is changed according to protocol
		(TCP state is changed to ESTABLISHED).
	* support for the netchannels and packet socket interface.

It was proven [3] that unetstack can be *much* faster than vanilla TCP
stack (mainly because of heavily reduced number of syscalls, different
congestion control algorithm and other features).

This release includes number of bugs fixed (thanks to Salvatore Del
Popolo for continuously kicking me), packet socket interface (enabled at
compile time) and small documentation update (including example).

Attached archive for intrested readers.

Thank you.

1. Userspace network stack.
http://tservice.net.ru/~s0mbre/old/?section=projects&item=unetstack

2. Netchannel subsystem.
http://tservice.net.ru/~s0mbre/old/?section=projects&item=netchannel

3. Gigabit send/recv benchmark for netchannels and sockets using small
packets.
http://tservice.net.ru/~s0mbre/blog/devel/networking/2006_10_26.html
http://tservice.net.ru/~s0mbre/blog/devel/networking/2006_12_21.html

-- 
	Evgeniy Polyakov

[-- Attachment #2: unetstack-2007_09_25.tar.gz --]
[-- Type: application/x-tar-gz, Size: 25717 bytes --]

Re: Userspace network stack and netchannels.

[Stephen Hemminger]

On Tue, 25 Sep 2007 22:00:48 +0400
Evgeniy Polyakov <johnpol@2ka.mipt.ru> wrote:

> Hi.
> 
> I've released new version of the extremely small 
> userspace netowork stack [1] implementation.
> Stack supports TCP and UDP over IP.
> It works on top of netchannels [2] or packet socket 
> (returned back in this release).
> 
> Supported features:
> 	* TCP/UDP sending and receiving.
> 	* Timestamp, window scaling, MSS TCP options.
> 	* PAWS.
> 	* Slow start and congestion control.
> 	* Socket-like interface.
> 	* Retransmit and fast retransmit support.
> 	* support for TCP listen state (only point-to-point mode,
> 		i.e. no new data channels are created, when new client is
> 		connected, instead state is changed according to protocol
> 		(TCP state is changed to ESTABLISHED).
> 	* support for the netchannels and packet socket interface.
> 
> It was proven [3] that unetstack can be *much* faster than vanilla TCP
> stack (mainly because of heavily reduced number of syscalls, different
> congestion control algorithm and other features).

I am glad to see a more complete implementation for comparison, but
by changing several variables at once it makes it hard to evaluate whether
the improvement comes from user space TCP or the other changes.

Re: Userspace network stack and netchannels.

[Evgeniy Polyakov]

On Tue, Sep 25, 2007 at 12:34:44PM -0700, Stephen Hemminger (shemminger@linux-foundation.org) wrote:
> > It was proven [3] that unetstack can be *much* faster than vanilla TCP
> > stack (mainly because of heavily reduced number of syscalls, different
> > congestion control algorithm and other features).
> 
> I am glad to see a more complete implementation for comparison, but
> by changing several variables at once it makes it hard to evaluate whether
> the improvement comes from user space TCP or the other changes.

Number of issues were improved - mostly reduced number of syscalls (if
size of the message increases results become essentially the same).
Different (non-rfc-compliant) congestion control algorithm allows faster
start and recovery, but in slower links (or longer rtt) it can be wrong.
Originally it was created to show superiority of the netchannel architecture,
but several hundreds of percents in some cases is not result of
peer-to-peer and process context processing. This stack works with
netchannels, which just became a link between hardware and user, such
approach allows huge variety of usage for different network hardware
including safe offloading, sniffing and so on. This stack is an example.

-- 
	Evgeniy Polyakov

Re: Userspace network stack and netchannels.

[Robert Iakobashvili]

Evgeniy,

On 9/25/07, Evgeniy Polyakov <johnpol@2ka.mipt.ru> wrote:
>  I've released new version of the extremely small
> userspace netowork stack [1] implementation.
>

>  1. Userspace network stack.
> http://tservice.net.ru/~s0mbre/old/?section=projects&item=unetstack
>
> 2. Netchannel subsystem.
>  http://tservice.net.ru/~s0mbre/old/?section=projects&item=netchannel
>
> 3. Gigabit send/recv benchmark for netchannels and sockets using small
> packets.
> http://tservice.net.ru/~s0mbre/blog/devel/networking/2006_10_26.html
>  http://tservice.net.ru/~s0mbre/blog/devel/networking/2006_12_21.html
>

Very interesting. Are there any details of your benchmarking available,
namely, how it was done?
Have you some comparison profiling data (oprofile) to figure out where
the advantages are coming from?

-- 
Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
 http://curl-loader.sourceforge.net
A web testing and traffic generation tool.

-- 
Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
A web testing and traffic generation tool.

Re: Userspace network stack and netchannels.

[Evgeniy Polyakov]

Hi Robert.

On Tue, Sep 25, 2007 at 09:20:44PM +0200, Robert Iakobashvili (coroberti@gmail.com) wrote:
> > 3. Gigabit send/recv benchmark for netchannels and sockets using small
> > packets.
> > http://tservice.net.ru/~s0mbre/blog/devel/networking/2006_10_26.html
> > http://tservice.net.ru/~s0mbre/blog/devel/networking/2006_12_21.html
> >
> 
> Very interesting. Are there any details of your benchmarking available,
> namely, how it was done?
> Have you some comparison profiling data (oprofile) to figure out where
> the advantages are coming from?

Those are throughput tests - one sender, one receiver with different
size of the packet being sent/received. CPU usage was determined via
/proc statistics. My analysis says that reduced number of syscalls is
main improvement factor, but different (non-rfc-compliant) congestion 
control allows faster start and faster recovery.

-- 
	Evgeniy Polyakov

Re: Userspace network stack and netchannels.

[Robert Iakobashvili]

On 9/26/07, Evgeniy Polyakov <johnpol@2ka.mipt.ru> wrote:
> > > http://tservice.net.ru/~s0mbre/blog/devel/networking/2006_10_26.html
> > > http://tservice.net.ru/~s0mbre/blog/devel/networking/2006_12_21.html
> > >
> >
> > Very interesting. Are there any details of your benchmarking available,
> > namely, how it was done?
> > Have you some comparison profiling data (oprofile) to figure out where
> > the advantages are coming from?

> My analysis says that reduced number of syscalls is
> main improvement factor,
Great! Just oprofile of the comparison tests is of my curiosity.

> but different (non-rfc-compliant) congestion
> control allows faster start and faster recovery.

People like compliance to standards, even if it decreases performance.
:) Take care.

-- 
Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
A web testing and traffic generation tool.

Re: Userspace network stack and netchannels.

[jamal]

Evgeniy,

Those numbers look impressive. How mysterious do things get in SMP?
Sorry havent had time to stare at the code. How do you measure cpu you
quoted?

cheers,
jamal

Re: Userspace network stack and netchannels.

[Evgeniy Polyakov]

Hi Jamal.

On Wed, Sep 26, 2007 at 09:08:45AM -0400, jamal (hadi@cyberus.ca) wrote:
> Those numbers look impressive. How mysterious do things get in SMP?
> Sorry havent had time to stare at the code. How do you measure cpu you
> quoted?

I did not checked how netchannels and userspace stack behaves on SMP
just because I do not have SMP test machines. Getting into account that
netchannels have smaller number of locks (actually only one to queue
skb) compared to sockets, it should not behave worse, but I did not test
that.

CPU usage was obtained via /proc/$pid statistics.

-- 
	Evgeniy Polyakov