Network Acceleration Module vs Service

Network Acceleration Module vs Service

[Justin Yaple]

Hi everyone,

I have been working on a network accelerator, and I have reached a
point that I think it would be wise to get some guidance in what
direction it should go with its development.  Probably should have
done this a while ago.  I started working on this because the company
I work for can not afford to implement any of the commercially
available network accelerators.  I also wanted something that would
function similar to those products, and would be part of the open
source community.

What I have so far is a kernel module that uses the netfilter hooks to
intercept all IP packets that are being forwarded/routed.  It uses
some custom TCP Options to automatically detect other hosts running
the module, and begin compressing TCP sessions that traverse through
two hosts running the module.  It has logic to prevent compressed
traffic from being received by the destination client should one of
the host accelerators crash, or the service be stopped in an
out-of-line configuration.  It also has the ability to remove dead
sessions from its session tracking list.

Currently it can only compress/decompress the data field of the TCP
segments.  I want to add the ability for it to generate signatures for
common data patterns, and then substitute the signature in place of
that data pattern.  The other accelerator host would the reverse that
and replace the signature with the actual data pattern.  This will
need to support full and partial meshed networks like MPLS, DMVPN,
Metro LAN, Frame Relay so it would need to track what signatures are
available between the host accelerators.

At this point I could not figure out any method to achieve this in the
kernel because the signatures would need to be saved until the
signature is deleted, and there would need to be some type of lookup
to match data patterns to hosts that have signatures for those
patterns.  The only way I know how to do something like that is with a
database, and database access is pretty must restricted to user space
only from what I have read.

I am a little worried about the performance of the system overall
assuming the only option is to move the majority of this to a user
space service I would need to rewrite the module portion to QUEUE the
packets to user space.  This might severely effect performance of the
system in the number of packets per second that can be processed.
Given that it could be doing database queries, and substituting TCP
data for a signature or compressing the data the time it takes to move
a packet from kernel to user space and back might be negligible.

I just want some confirmation that I am on the right track before I
start converting this to a user space service.  I can email the source
for the module if anyone is interested in seeing it.  Its pretty ugly
compared to what I have seen here, but I am learning a lot by jumping
in, and just doing it.

Thanks,
Justin Yaple

Re: Network Acceleration Module vs Service

[Jan Engelhardt]

On Friday 2010-04-02 02:16, Justin Yaple wrote:
>
>I have been working on a network accelerator, [...]
>Currently it can only compress/decompress the data field of the TCP
>segments.

So, what makes it better than ipcomp?

>I want to add the ability for it to generate signatures for
>common data patterns, and then substitute the signature in place of
>that data pattern.  The other accelerator host would the reverse that
>and replace the signature with the actual data pattern.

That is essentially the definition of coding/compression,
which you already seem to have (minus one change that you don't
throw away the dictionary).

>I am a little worried about the performance of the system overall
>assuming the only option is to move the majority of this to a user
>space service I would need to rewrite the module portion to QUEUE the
>packets to user space.  This might severely effect performance of the
>system in the number of packets per second that can be processed.
>Given that it could be doing database queries, and substituting TCP
>data for a signature or compressing the data the time it takes to move
>a packet from kernel to user space and back might be negligible.

If it is not the case already, devise a mechanism so that userspace gathers
multiple nfqueued packets at once instead of one-at-a-time.
 - multi-threading the userspace process might be worth looking into
 - using NFQUEUE to deliver into separate queues per CPU
   (so that one CPU does not have to wait for the lock of another)

Re: Network Acceleration Module vs Service

[Justin Yaple]

> So, what makes it better than ipcomp?

At this point probably nothing, but it is a little different.  After
reading the RFC it looks like ipcomp compresses the entire IP packet
data field while this only compresses TCP segments data field leaving
the majority of the TCP header alone. I do modify the TCP options a
bit.  I looked up how to setup ipcomp, and correct me if I am wrong it
seems to only work on VPN tunnels while this can compress/decompress
any traffic the host is forwarding so it is mesh friendly for MPLS,
Frame Relay, and other meshed WANs.

> That is essentially the definition of coding/compression,
> which you already seem to have (minus one change that you don't
> throw away the dictionary).

Maybe I was not very clear in what these signatures would represent.
They would not be common patterns within the segment data field, but
they would represent an entire TCP segment data field.  When a TCP
segment is sent between two of these hosts they would copy the data
field to the drive, and generate a 40 byte signature composed of a 32
byte SHA-256 base signature, and a 8 byte unique ID.  So a 1400 byte
payload would be replaced by a 40 byte signature when that same data
is transmitted again.  Its effectively substituting the data field for
the signature.  Of course you cannot create signatures for every
possible data field so it would have to use a system to count commonly
sent data, and keep those while allowing the less common ones to be
removed from the system.

> If it is not the case already, devise a mechanism so that userspace gathers
> multiple nfqueued packets at once instead of one-at-a-time.
>  - multi-threading the userspace process might be worth looking into
>  - using NFQUEUE to deliver into separate queues per CPU
>   (so that one CPU does not have to wait for the lock of another)

That is exactly what I was thinking, but was not sure if it was a good
idea or not as I have never tried anything like this.  I was thinking
of using one thread to gather the packets, and assign them to queues
each queue would be processed by a separate worker thread per CPU core
- 1 if there are more than one core.  The thread for gathering the
packets would share the CPU with the rest of the system, and other
processes that might be implemented such as a WCCPv2 client.
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html