ip_rt_bug in mangle/OUTPUT

ip_rt_bug in mangle/OUTPUT

[Rennie deGraaf]

[-- Attachment #1: Type: text/plain, Size: 3921 bytes --]

I seem to be getting the error message
    ip_rt_bug: 10.1.1.1 -> 10.0.1.2, ?
whenever I attempt to send a packet with a non-local source IP address
(my local IP address is 10.0.1.2) from libipq in mangle/OUTPUT.  I have
observed this behaviour under Linux kernels 2.6.20.7 and
2.6.18-1.2257.fc5smp (Fedora Core 5), and iptables versions 1.3.5 and
1.3.7.

I'm trying to simulate connections with remote hosts by redirecting
packets to servers listening on localhost.  My strategy is to send
packets to IP_QUEUE from rules in the mangle/OUTPUT chain: destination
addresses are re-written on packets that I want to redirect, source
addresses are re-written on packets on responses to redirected packets,
and other packets are passed without modification.  A simplified, highly
stripped down version of my program is attached.

To run my example program, you need rules in your mangle/OUTPUT chain
forwarding packets to 10.1.1.1:123/TCP and from 127.0.0.1:22/TCP to
QUEUE, and something listening on 127.0.0.1:22/TCP.  If it worked
properly, a connection could be successfully established to
127.0.0.1:22/TCP by connecting to 10.1.1.1:123/TCP (using telnet, for
instance).

Do any of the gurus on this list know how I might fix or work around
this issue?

This issue seems to have been discussed before (such as
http://www.ussg.iu.edu/hypermail/linux/kernel/0504.3/0159.html), but
doesn't seem to have been resolved.

Thanks,
Rennie deGraaf


-----------------
#include <linux/netfilter.h>
#include <libipq.h>
#include <stdio.h>
#include <netinet/ip.h>
#include <netinet/tcp.h>
#include <arpa/inet.h>


u_int16_t ip_checksum(u_int32_t init, const u_int8_t* buf, size_t len)
{
    u_int32_t sum = init;
    u_int16_t* shorts = (u_int16_t*)buf;

    while (len > 1)
    {
        sum += *shorts++;
        len -= 2;
    }

    if (len == 1)
        sum += *(u_int8_t*)shorts;

    while (sum >> 16)	
        sum = (sum >> 16) + (sum & 0xFFFF);

    return ~sum;
}

u_int16_t tcp_checksum(const struct iphdr* iph, const struct tcphdr*
tcph, size_t len)
{
    u_int32_t cksum = 0;

    cksum += (iph->saddr >> 16) & 0x0000ffff;
    cksum += iph->saddr & 0x0000ffff;
    cksum += (iph->daddr >> 16) & 0x0000ffff;
    cksum += iph->daddr & 0x0000ffff;
    cksum += htons(iph->protocol & 0x00ff);
    cksum += htons(len);
    return ip_checksum(cksum, (unsigned char*)tcph, len);
}


void handle_packet(unsigned char* pkt, size_t len)
{
    struct iphdr* iph = (struct iphdr*) pkt;
    struct tcphdr* tcph = (struct tcphdr*)(pkt+iph->ihl*4);

    if ((iph->daddr == htonl(0x0a010101)) && (tcph->dest == htons(123)))
    {
        printf("forward\n");
        iph->daddr = htonl(0x7f000001);
        tcph->dest = htons(22);

        iph->check = 0;
        iph->check = ip_checksum(0, pkt, iph->ihl*4);
        tcph->check = 0;
        tcph->check = tcp_checksum(iph, tcph, len-(iph->ihl*4));
    }
    else if ((iph->saddr == htonl(0x7f000001)) && (tcph->source ==
htons(22)))
    {
        printf("reverse\n");
        iph->saddr = htonl(0x0a010101);
        tcph->source = htons(123);

        iph->check = 0;
        iph->check = ip_checksum(0, pkt, iph->ihl*4);
        tcph->check = 0;
        tcph->check = tcp_checksum(iph, tcph, len-(iph->ihl*4));
    }
    else
        printf("wrong packet!\n");
}

int main()
{
    struct ipq_handle* handle;
    unsigned char buf[10000];
    ipq_packet_msg_t* pkt;

    handle = ipq_create_handle(0, PF_INET);
    ipq_set_mode(handle, IPQ_COPY_PACKET, 65535);

    while (1)
    {
        ipq_read(handle, buf, 10000, 0);
        pkt = ipq_get_packet(buf);
        handle_packet(pkt->payload, pkt->data_len);
        ipq_set_verdict(handle, pkt->packet_id, NF_ACCEPT,
pkt->data_len, pkt->payload);
    }

    ipq_destroy_handle(handle);
    return 0;
}


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: ip_rt_bug in mangle/OUTPUT

[Patrick McHardy]

Rennie deGraaf wrote:
> I seem to be getting the error message
>     ip_rt_bug: 10.1.1.1 -> 10.0.1.2, ?
> whenever I attempt to send a packet with a non-local source IP address
> (my local IP address is 10.0.1.2) from libipq in mangle/OUTPUT.  I have
> observed this behaviour under Linux kernels 2.6.20.7 and
> 2.6.18-1.2257.fc5smp (Fedora Core 5), and iptables versions 1.3.5 and
> 1.3.7.
> 
> I'm trying to simulate connections with remote hosts by redirecting
> packets to servers listening on localhost.  My strategy is to send
> packets to IP_QUEUE from rules in the mangle/OUTPUT chain: destination
> addresses are re-written on packets that I want to redirect, source
> addresses are re-written on packets on responses to redirected packets,
> and other packets are passed without modification.  A simplified, highly
> stripped down version of my program is attached.
> 
> To run my example program, you need rules in your mangle/OUTPUT chain
> forwarding packets to 10.1.1.1:123/TCP and from 127.0.0.1:22/TCP to
> QUEUE, and something listening on 127.0.0.1:22/TCP.  If it worked
> properly, a connection could be successfully established to
> 127.0.0.1:22/TCP by connecting to 10.1.1.1:123/TCP (using telnet, for
> instance).
> 
> Do any of the gurus on this list know how I might fix or work around
> this issue?


If you don't need the rerouting to be happen (you only change the
source address and don't use routing rules based on that) you can
simply return NF_STOP instead of NF_ACCEPT. It will do exactly
the same thing but avoid rerouting.

> This issue seems to have been discussed before (such as
> http://www.ussg.iu.edu/hypermail/linux/kernel/0504.3/0159.html), but
> doesn't seem to have been resolved.


The solution suggested there doesn't work since we've moved
the okfn invocation to the caller of nf_hook_slow() and I'm
hoping we can some day get rid of the okfn function pointer
entirely.

Re: ip_rt_bug in mangle/OUTPUT

[Rennie deGraaf]

[-- Attachment #1: Type: text/plain, Size: 730 bytes --]

Patrick McHardy wrote:
> Rennie deGraaf wrote:
>> I seem to be getting the error message
>>     ip_rt_bug: 10.1.1.1 -> 10.0.1.2, ?
>> whenever I attempt to send a packet with a non-local source IP address
>> (my local IP address is 10.0.1.2) from libipq in mangle/OUTPUT.  I have
>> observed this behaviour under Linux kernels 2.6.20.7 and
>> 2.6.18-1.2257.fc5smp (Fedora Core 5), and iptables versions 1.3.5 and
>> 1.3.7.
> 
> If you don't need the rerouting to be happen (you only change the
> source address and don't use routing rules based on that) you can
> simply return NF_STOP instead of NF_ACCEPT. It will do exactly
> the same thing but avoid rerouting.

Thanks; that did the trick.

Rennie deGraaf


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: ip_rt_bug in mangle/OUTPUT

[Rennie deGraaf]

[-- Attachment #1: Type: text/plain, Size: 2094 bytes --]

Patrick McHardy wrote:
> Rennie deGraaf wrote:
>> I seem to be getting the error message
>>     ip_rt_bug: 10.1.1.1 -> 10.0.1.2, ?
>> whenever I attempt to send a packet with a non-local source IP address
>> (my local IP address is 10.0.1.2) from libipq in mangle/OUTPUT.  I have
>> observed this behaviour under Linux kernels 2.6.20.7 and
>> 2.6.18-1.2257.fc5smp (Fedora Core 5), and iptables versions 1.3.5 and
>> 1.3.7.
>>
>> I'm trying to simulate connections with remote hosts by redirecting
>> packets to servers listening on localhost.  My strategy is to send
>> packets to IP_QUEUE from rules in the mangle/OUTPUT chain: destination
>> addresses are re-written on packets that I want to redirect, source
>> addresses are re-written on packets on responses to redirected packets,
>> and other packets are passed without modification.  A simplified, highly
>> stripped down version of my program is attached.
>>
>> To run my example program, you need rules in your mangle/OUTPUT chain
>> forwarding packets to 10.1.1.1:123/TCP and from 127.0.0.1:22/TCP to
>> QUEUE, and something listening on 127.0.0.1:22/TCP.  If it worked
>> properly, a connection could be successfully established to
>> 127.0.0.1:22/TCP by connecting to 10.1.1.1:123/TCP (using telnet, for
>> instance).
>>
>> Do any of the gurus on this list know how I might fix or work around
>> this issue?
> 
> 
> If you don't need the rerouting to be happen (you only change the
> source address and don't use routing rules based on that) you can
> simply return NF_STOP instead of NF_ACCEPT. It will do exactly
> the same thing but avoid rerouting.

That solution worked well on recent kernels.  Unfortunately, my boss now
wants my code to work on Linux 2.6.9, which doesn't appear to have
NF_STOP.  (It seems to have been added in 2.6.12.)  Can you think of any
other work-arounds, short of dropping the packets and re-injecting the
modified versions through raw sockets?

In the meantime, I'll try to convince my boss to upgrade to a more
modern kernel.

Thanks,
Rennie deGraaf


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: ip_rt_bug in mangle/OUTPUT

[Patrick McHardy]

Rennie deGraaf wrote:
> Patrick McHardy wrote:
> 
>>If you don't need the rerouting to be happen (you only change the
>>source address and don't use routing rules based on that) you can
>>simply return NF_STOP instead of NF_ACCEPT. It will do exactly
>>the same thing but avoid rerouting.
> 
> 
> That solution worked well on recent kernels.  Unfortunately, my boss now
> wants my code to work on Linux 2.6.9, which doesn't appear to have
> NF_STOP.  (It seems to have been added in 2.6.12.)  Can you think of any
> other work-arounds, short of dropping the packets and re-injecting the
> modified versions through raw sockets?


No, old kernel version will even leak packets when you send unknown
return codes.