Caching packets with libipq

Caching packets with libipq

[David Murray]

Hi,

I am trying to build a TCP proxy that is capable of reordering out of 
order TCP segments before delivering them to the end machine. I plan on 
doing this by analyzing the TCP sequence numbers but for now I am just 
trying to get familiar with netfilter.

I am currently writing this using libipq. The reason that I am not 
using the newer libnetfilter is because there seems to be way more 
documentation and examples for libipq rather than libnetfilter. Using 
libipq I am able to print out packet details and NF_ACCEPT or NF_DROP, 
however, the problem I am having is with copying and storing packets.

The program plan is to anticipate the next sequence number for a each 
flow so: nxt_seq_num  = curr_pkt_seq_num + curr_pkt_data_payload. If I 
detect that one packet is missing I want to buffer up to ten packets 
waiting for the delayed packet.

If a segment is missing from the sequence I want to copy packets into 
memory and NF_DROP until the correct segment arrives. We can then 
replay our buffered packets, and the TCP receiver will receive the 
packets in order.

Currently, I am just trying to see if this is possible in libipq 
because I have not found any similar examples of caching packets in 
libipq. So, in the code snippet below I am trying to cache the 50th 
packet, and then NF_DROP it from the queue. The program should then 
NF_ACCEPT packet 51 and inject packet 50. So if the program was working 
correctly, I would be rearranging the sequence of frames such that 
packets would arrive: 48, 49, 51, 50, 52

The problem is that it appears that packet 50 is never being replayed 
and from the terminal I am getting  a ?Received error message 2?. I 
have been unable to get further information on what this error message 
means. Interestingly, when replaying the cached packet I am still able 
to print out information from it, eg TCP sequence number and 
destination IP address.

Can anyone provide any advice as to how I can narrow this problem down 
further.

Thanks for your time
Dave

status = ipq_set_mode(h, IPQ_COPY_PACKET, BUFSIZE);
     if (status < 0)
    die(h);
        do {
    status = ipq_read(h, buf, BUFSIZE, 0);
    if (status < 0)
      die(h);
                  switch (ipq_message_type(buf)) {
                 case NLMSG_ERROR: {
        fprintf(stderr, "Received error message %d\n",
        ipq_get_msgerr(buf));
        break;
        }

        case IPQM_PACKET: /* got a packet */ {

        ipq_packet_msg_t *m = ipq_get_packet(buf);
                       struct iphdr *ip = (struct iphdr*) m->payload;
        struct tcphdr *tcp = (struct tcphdr*) (m->payload + (4 * ip->ihl));

        int unsigned payload_length = (unsigned int) ntohs(ip->tot_len) 
- ((ip->ihl << 2) + (tcp->doff << 2));
                       if (payload_length > 0) {
                         if (count != 50) {

            printf("This is frame %d dest port %u and seq num %u, size 
%d\n", count, htons(tcp->dest), ntohl(tcp->seq), m->data_len);
            status = ipq_set_verdict(h, m->packet_id, NF_ACCEPT, 0, NULL);
                    }
                           if (count == 50) {

            printf("Not sending 50th packet\n");
            printf("The frame was %d dest port %u and seq num %u, size 
%d\n", count, htons(tcp->dest), ntohl(tcp->seq), m->data_len);
            printf("Chaching instead!\n");

            free(store);            store = (ipq_packet_msg_t 
*)calloc(1, BUFSIZE);
             store->packet_id = m->packet_id;        /* ID of queued packet */
            store->mark = m->mark;                  /* Netfilter mark value */
            store->timestamp_sec = m->timestamp_sec;               
store->timestamp_usec = m->timestamp_usec;                store->hook = 
m->hook;           /* Netfilter hook we rode in on */
            store->hw_protocol = m->hw_protocol;              
store->hw_type = m->hw_type;       /* Hardware type */
            store->data_len = m->data_len;         /* Length of packet data */
            memcpy(store->payload, m->payload, m->data_len);

            status = ipq_set_verdict(h, m->packet_id, NF_DROP, 0, NULL);
            printf("Status: %d\n", status);             }
           if (count == 51) {

            struct iphdr *ip = (struct iphdr*) store->payload;
            struct tcphdr *tcp = (struct tcphdr*) (store->payload + (4 
* ip->ihl));
            printf("Replaying missed frame 50 dest port %u and seq num 
%u, size %d\n", htons(tcp->dest), ntohl(tcp->seq), store->data_len);

            printf("Payload: %s\n", store->payload);
            status = ipq_set_verdict(h, store->packet_id, NF_ACCEPT, 
store->data_len, store->payload);
            printf("Status: %d\n", status);
            }

          count++;
          }

        else {
          status = ipq_set_verdict(h, m->packet_id, NF_ACCEPT, 0, NULL);
          }

        break;
        }
                             default:
        fprintf(stderr, "Unknown message type!\n");
        break;
      }     } while (1);

Re: Caching packets with libipq

[Eric Leblond]

H,

Le vendredi 16 janvier 2009 à 10:41 +0900, David Murray a écrit :
> Hi,
>
> I am currently writing this using libipq. The reason that I am not 
> using the newer libnetfilter is because there seems to be way more 
> documentation and examples for libipq rather than libnetfilter. Using 
> libipq I am able to print out packet details and NF_ACCEPT or NF_DROP, 
> however, the problem I am having is with copying and storing packets.

What about:
	http://www.nufw.org/doc/libnetfilter_queue/

Real question: what is missing here ?

BR,
-- 
Eric Leblond <eleblond@inl.fr>
INL: http://www.inl.fr/
NuFW: http://www.nufw.org/
EdenWall: http://www.edenwall.com/

--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Caching packets with libipq

[David Murray]

Eric Leblond wrote:
>
> What about:
> 	http://www.nufw.org/doc/libnetfilter_queue/
>
> Real question: what is missing here ?
>
> BR,
>   
Thanks for the reply Eric. In my own opinion (things that could be done 
to improve that documentation:

1. Navigation seems a little cluttered. I think it would be better to 
just have a link to documentation on the first page and then a link on 
every subsequent page to page x + 1.

2. On the main page it would be nice to have a description of why 
libnetfilter_queue is better than libipq

3. The page needs to be easier to find. Why is there not a link to it on 
the netfilter site?

4. You don't specify the compile time commands that must be used with 
libnet filter -lnfnetlink -lnetfilter_queue

5. The example code you give includes a callback function. Can you write 
a libnetfilter_queue program without a call back? It might be easier for 
people without lots of c experience to modify and start doing things.

All of these suggestions might not be correct / easy to implement. It is 
just my opinion that they might improve the site.

Thanks for your time
Dave

Re: Caching packets with libipq

[Pablo Neira Ayuso]

David Murray wrote:
> Eric Leblond wrote:
>>
>> What about:
>>     http://www.nufw.org/doc/libnetfilter_queue/
>>
>> Real question: what is missing here ?
>>
>> BR,
>>   
> Thanks for the reply Eric. In my own opinion (things that could be done
> to improve that documentation:
> 
> 1. Navigation seems a little cluttered. I think it would be better to
> just have a link to documentation on the first page and then a link on
> every subsequent page to page x + 1.
> 
> 2. On the main page it would be nice to have a description of why
> libnetfilter_queue is better than libipq
> 
> 3. The page needs to be easier to find. Why is there not a link to it on
> the netfilter site?

There will be one soon, I have no time to upload it. The patch to
convert documentation barely hit the git tree weeks ago. There was no
documentation months ago, so we're getting better in this field ;)

> 4. You don't specify the compile time commands that must be used with
> libnet filter -lnfnetlink -lnetfilter_queue
> 
> 5. The example code you give includes a callback function. Can you write
> a libnetfilter_queue program without a call back? It might be easier for
> people without lots of c experience to modify and start doing things.

IIRC nfq_handle_packet uses that callback internally.

> All of these suggestions might not be correct / easy to implement. It is
> just my opinion that they might improve the site.

They are welcome.

-- 
"Los honestos son inadaptados sociales" -- Les Luthiers

Re: Caching packets with libipq

[David Murray]

Quoting Pablo Neira Ayuso <pablo@netfilter.org>:

>> 5. The example code you give includes a callback function. Can you write
>> a libnetfilter_queue program without a call back? It might be easier for
>> people without lots of c experience to modify and start doing things.
>
> IIRC nfq_handle_packet uses that callback internally.
>

Are you saying that libnetfilter_queue can be used without using 
nfq_create_queue. As far as I can tell (FYI I am a c novice), if you 
use nfq_create_queue it must contain the a link to the address of the 
callback function. All of the examples I can find do all the packet 
processing within this callback function. If this is the case, I cannot 
understand how you could keep ongoing statistics of packets or flows 
via libnetfilter_queue without using a lot of globals.