Facing trouble in creating a packet in kernel space

Facing trouble in creating a packet in kernel space

[Rifat Rahman]

Hello there,

I need to mangle rtp packets in kernel space. So far I am new in kernel
module programming. I am trying to implement a module for netfilter hooks.
For the first time as exercise, I am trying to write smaller modules. Let
me explain what I am actually doing now.

I have an echo client and server. The server runs on port 6000. Both are on
different machines (May be VMs in bridge filter mode). The client sends udp
message and the server just echoes it back. Let us suppose the client sends
"some message" as data. Then now I am trying to write a module for the
client machine that will append "12345" after the data so that the server
will get "some message12345" and echo it back. Now there are various things
I did faced. I relied on the NF_IP_POST_ROUTING hook.

At first, I copied the data to a temporary storage, and then add 12345 with
that. Then I increase skb->tail using skb_put(). Then I memset()  0 to the
packet data, and copy the temporary storage with that. Then as the
procedure, NF_ACCEPT is returned. There are certain checking points like
the udph->dest == 6000 etc. etc. When I use skb_put(), my system hangs out
after two or three minutes. When I dmesg to be certain that everything goes
right, I find it OK. But, suppose once I send a message like "This is a
pretty big message" and another time I send "small message" then I get just
"small message12345g message" that means, the bigger message is stored
somewhere I don't understand. I tried with skb_add_data() but that works
incorrectly here, I understand it's my fault. I just can't figure it out.

Now, one thing came in my mind, if it's not possible, should I create new
packets for that data appending? I find skb->end - skb->tail is not so big.
But ultimately I have to merge two or three packets into one packet and
then skb_put() will not suffice for me. Then the point comes, I can use
alloc_skb(), skb_reserve(), skb_header_pointer() and other skb manipulation
functions, but I don't understand how can I drop the packet got (should I
return NF_DROP?) and how can I route my created packets in the packet
flowing path? I saw in the xtables_addons for help (file xt_ECHO.c) but
found they used ip_route_me_harder() for it. But the function they used is
quite different, it is their own. Then I need suggestions about how to
route the newly created packet in kernel space. This is the point I am
stuck for a few days. If anyone can help me, I will be highly grateful. I
understand my knowledge is poor about Linux kernel, but I need a definite
way to understand, otherwise the code is not helping me much.

Thanks in advance.

-- 
Rifat Rahman
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120925/9892d93f/attachment.html 

Facing trouble in creating a packet in kernel space

[michi1 at michaelblizek.twilightparadox.com]

Hi!

On 16:28 Tue 25 Sep     , Rifat Rahman wrote:
> Hello there,
> 
> I need to mangle rtp packets in kernel space. So far I am new in kernel
> module programming. I am trying to implement a module for netfilter hooks.
> For the first time as exercise, I am trying to write smaller modules. Let
> me explain what I am actually doing now.
> 
> I have an echo client and server. The server runs on port 6000. Both are on
> different machines (May be VMs in bridge filter mode). The client sends udp
> message and the server just echoes it back. Let us suppose the client sends
> "some message" as data. Then now I am trying to write a module for the
> client machine that will append "12345" after the data so that the server
> will get "some message12345" and echo it back. Now there are various things
> I did faced. I relied on the NF_IP_POST_ROUTING hook.

I do not understand why you try to do this in the kernel at all. Why does the
client app not just send "some message12345" itself? If you want to mange the
data in transit, why not use a transparent proxy instead?
http://stackoverflow.com/questions/5615579/how-to-get-original-destination-port-of-redirected-udp-message

> At first, I copied the data to a temporary storage, and then add 12345 with
> that. Then I increase skb->tail using skb_put(). Then I memset()  0 to the
> packet data, and copy the temporary storage with that. Then as the
> procedure, NF_ACCEPT is returned. There are certain checking points like
> the udph->dest == 6000 etc. etc. When I use skb_put(), my system hangs out
> after two or three minutes.

What does it do exactly? If you do skb_put() and there is no space, you should
get something like "skb_over_panic".

> When I dmesg to be certain that everything goes
> right, I find it OK. But, suppose once I send a message like "This is a
> pretty big message" and another time I send "small message" then I get just
> "small message12345g message" that means, the bigger message is stored
> somewhere I don't understand. I tried with skb_add_data() but that works
> incorrectly here, I understand it's my fault. I just can't figure it out.

Could it be that the small message happened to allocate the same memory the
previous packet used and thus has some unallocated data at the end?

> Now, one thing came in my mind, if it's not possible, should I create new
> packets for that data appending? I find skb->end - skb->tail is not so big.

You might have to do so in some cases. But it might have some side effects
nobody would think about. For example, take a look at this:
http://lxr.linux.no/#linux+v3.5.4/net/sched/cls_cgroup.c#L117
It essentially means that the packet queue layer2 accesses data all the way up
to the socket layer. If you just copy the data, this will break. More things
like this may exist.

You might be also able to allocate a larger buffer and reuse the sk_buff. It
might be less painful.

> But ultimately I have to merge two or three packets into one packet and
> then skb_put() will not suffice for me. Then the point comes, I can use
> alloc_skb(), skb_reserve(), skb_header_pointer() and other skb manipulation
> functions, but I don't understand how can I drop the packet got (should I
> return NF_DROP?)

There should be a way to drop packets inside netfilter rules (maybe not in
postrouting tough). I did not look into the code right now. Why not try
returning NF_DROP and see if it leaks?

> and how can I route my created packets in the packet flowing path?

You could do it the dirty way and just call dev_queue_xmit(). The packet will
be directly sent to the device without going through all hook (including yours)
a second time. You have to be careful about the udp checksum and fragmentation.
Also, if ipsec is in use, it will most likely by-passed.

There might be better interfaces for this. I did not look into it closely
right now. 

	-Michi
-- 
programing a layer 3+4 network protocol for mesh networks
see http://michaelblizek.twilightparadox.com

Facing trouble in creating a packet in kernel space

[Peter Teoh]

Yes, Michael has a point.   proxy is easier than kernel.    I used
Webscarab for this.   Alternatively, another tool I used is scapy (no
proxy setup is needed).   And I must say it is a FANTASTIC tool for
this purpose.   First u capture with wireshark, and then replay via
scapy, which has a function called "fuzz()" for this purpose.

http://www.secdev.org/conf/scapy_pacsec05.handout.pdf

http://media.packetlife.net/media/library/36/scapy.pdf

http://theitgeekchronicles.files.wordpress.com/2012/05/scapyguide1.pdf

it is low level enough, for you to fuzz at different protocol inside
each packet.

On Wed, Sep 26, 2012 at 2:43 AM,
<michi1@michaelblizek.twilightparadox.com> wrote:
> Hi!
>
> On 16:28 Tue 25 Sep     , Rifat Rahman wrote:
>> Hello there,
>>
>> I need to mangle rtp packets in kernel space. So far I am new in kernel
>> module programming. I am trying to implement a module for netfilter hooks.
>> For the first time as exercise, I am trying to write smaller modules. Let
>> me explain what I am actually doing now.
>>
>> I have an echo client and server. The server runs on port 6000. Both are on
>> different machines (May be VMs in bridge filter mode). The client sends udp
>> message and the server just echoes it back. Let us suppose the client sends
>> "some message" as data. Then now I am trying to write a module for the
>> client machine that will append "12345" after the data so that the server
>> will get "some message12345" and echo it back. Now there are various things
>> I did faced. I relied on the NF_IP_POST_ROUTING hook.
>
> I do not understand why you try to do this in the kernel at all. Why does the
> client app not just send "some message12345" itself? If you want to mange the
> data in transit, why not use a transparent proxy instead?
> http://stackoverflow.com/questions/5615579/how-to-get-original-destination-port-of-redirected-udp-message
>
>> At first, I copied the data to a temporary storage, and then add 12345 with
>> that. Then I increase skb->tail using skb_put(). Then I memset()  0 to the
>> packet data, and copy the temporary storage with that. Then as the
>> procedure, NF_ACCEPT is returned. There are certain checking points like
>> the udph->dest == 6000 etc. etc. When I use skb_put(), my system hangs out
>> after two or three minutes.
>
> What does it do exactly? If you do skb_put() and there is no space, you should
> get something like "skb_over_panic".
>
>> When I dmesg to be certain that everything goes
>> right, I find it OK. But, suppose once I send a message like "This is a
>> pretty big message" and another time I send "small message" then I get just
>> "small message12345g message" that means, the bigger message is stored
>> somewhere I don't understand. I tried with skb_add_data() but that works
>> incorrectly here, I understand it's my fault. I just can't figure it out.
>
> Could it be that the small message happened to allocate the same memory the
> previous packet used and thus has some unallocated data at the end?
>
>> Now, one thing came in my mind, if it's not possible, should I create new
>> packets for that data appending? I find skb->end - skb->tail is not so big.
>
> You might have to do so in some cases. But it might have some side effects
> nobody would think about. For example, take a look at this:
> http://lxr.linux.no/#linux+v3.5.4/net/sched/cls_cgroup.c#L117
> It essentially means that the packet queue layer2 accesses data all the way up
> to the socket layer. If you just copy the data, this will break. More things
> like this may exist.
>
> You might be also able to allocate a larger buffer and reuse the sk_buff. It
> might be less painful.
>
>> But ultimately I have to merge two or three packets into one packet and
>> then skb_put() will not suffice for me. Then the point comes, I can use
>> alloc_skb(), skb_reserve(), skb_header_pointer() and other skb manipulation
>> functions, but I don't understand how can I drop the packet got (should I
>> return NF_DROP?)
>
> There should be a way to drop packets inside netfilter rules (maybe not in
> postrouting tough). I did not look into the code right now. Why not try
> returning NF_DROP and see if it leaks?
>
>> and how can I route my created packets in the packet flowing path?
>
> You could do it the dirty way and just call dev_queue_xmit(). The packet will
> be directly sent to the device without going through all hook (including yours)
> a second time. You have to be careful about the udp checksum and fragmentation.
> Also, if ipsec is in use, it will most likely by-passed.
>
> There might be better interfaces for this. I did not look into it closely
> right now.
>
>         -Michi
> --
> programing a layer 3+4 network protocol for mesh networks
> see http://michaelblizek.twilightparadox.com
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies



-- 
Regards,
Peter Teoh