trouble with expect_related

trouble with expect_related

[Rich Wellner]

I'm writing a module for ipfilters to deal with a protocol we use at work.
You'll pardon me for being long winded, but I'm not positive what's going
wrong.

The init works:

static int __init init(void)
{
  int retVal;
  memset(&dccp, 0, sizeof(struct ip_conntrack_helper));

  printk("ip_conntrack_dccp: registered\nbuild: %s, %s\n", __DATE__, __TIME__);
       
  dccp.tuple.src.u.tcp.port = htons(DCCP_PORT);
  dccp.tuple.dst.protonum = IPPROTO_TCP;
  dccp.mask.src.u.tcp.port = 0xFFFF;
  dccp.mask.dst.protonum = 0xFFFF;
  dccp.help = dccp_help;

  retVal = ip_conntrack_helper_register(&dccp);
	printk("retVal: %d\n", retVal);
	return retVal;
}

I claim it works because i see log messages and the helper is called as
expected.

My problem is with the helper.  I parse the packet out and run
ip_conntrack_expect_related but the incoming connection doesn't get passed
through.  I have verified using tcpdump that the connection request does come
from the remote machine, but my module doesn't seem to have registered the
return right.  Anyway, here's the helper (more commentary follows code):

static int dccp_help(const struct iphdr *iph, size_t len, 
                struct ip_conntrack *ct, 
                enum ip_conntrack_info ctinfo)
{
  struct tcphdr *tcph = (void *)iph + iph->ihl * 4;
  const char *data = (const char *)tcph + tcph->doff * 4;
  const char *_data = data;
  char *data_limit;
  u_int32_t tcplen = len - iph->ihl * 4;
  u_int32_t datalen = tcplen - tcph->doff * 4;
  int dir = CTINFO2DIR(ctinfo);
  struct ip_conntrack_tuple t, mask;
  u_int32_t dccp_ip;
  u_int16_t dccp_port;

  struct ip_ct_irc *info = &ct->help.ct_irc_info;

  /* Can't track connections formed before we registered */
  if (!info)
    return NF_ACCEPT;

  /* If packet is coming from IRC server */
  if (dir == IP_CT_DIR_REPLY)
    return NF_ACCEPT;

  /* Until there's been traffic both ways, don't look in packets. */
  if (ctinfo != IP_CT_ESTABLISHED
      && ctinfo != IP_CT_ESTABLISHED + IP_CT_IS_REPLY) {
    printk("Conntrackinfo = %u\n", ctinfo);
    return NF_ACCEPT;
  }

  /* Not whole TCP header? */
  if (tcplen < sizeof(struct tcphdr) || tcplen < tcph->doff * 4) {
    printk("tcplen = %u\n", (unsigned) tcplen);
    return NF_ACCEPT;
  }

  /* Checksum invalid?  Ignore. */
  /* FIXME: Source route IP option packets --RR */
  if (tcp_v4_check(tcph, tcplen, iph->saddr, iph->daddr,
                   csum_partial((char *) tcph, tcplen, 0))) {
    printk("bad csum: %p %u %u.%u.%u.%u %u.%u.%u.%u\n",
		     tcph, tcplen, NIPQUAD(iph->saddr),
		     NIPQUAD(iph->daddr));
    return NF_ACCEPT;
  }

  if( strstr(data, "open") ) {
    printk("open packet: %s", data);

  mask = ((struct ip_conntrack_tuple)
		{ { 0xFFFFFFFF, { 0 } },
		  { 0xFFFFFFFF, { 0xFFFF }, 0xFFFF }});

  parse_command(data, &dccp_ip, &dccp_port );

  printk("recieving socket is %d:%d\n", dccp_ip, dccp_port);

  LOCK_BH(&ip_dccp_lock);

  memset(&t, 0, sizeof(t));
  t.src.ip = ct->tuplehash[!dir].tuple.src.ip;
  t.src.u.tcp.port = 0;
  t.dst.ip = ct->tuplehash[IP_CT_DIR_REPLY].tuple.dst.ip;
  t.dst.u.tcp.port = htons(dccp_port);
  t.dst.protonum = IPPROTO_TCP;

  printk("expect_related %u.%u.%u.%u:%u-%u.%u.%u.%u:%u\n",
         NIPQUAD(t.src.ip),
         ntohs(t.src.u.tcp.port),
         NIPQUAD(t.dst.ip),
         ntohs(t.dst.u.tcp.port));

  ip_conntrack_expect_related(ct, &t, &mask, NULL);
  UNLOCK_BH(&ip_dccp_lock);
  }

  return NF_ACCEPT;
}

Anyone capable of helping me will probably recognize the general structure of
the above code as being that from the existing ftp/irc modules.  I tried to
use the doc on netfilter.org, but it seems to be pretty outdated, making, for
example, references to structure elements that don't exist anymore.

Significantly the last printk above shows what I expect to see: 
Jan  3 17:45:54 oxygen kernel: expect_related 131.225.80.34:0-207.252.250.2:38344

but the incoming connection, as i say above, never makes it through.

I've tried several different variations, but would be willing to try more if
people have only suggestions but no answers.  I'm just a bit out of my wits
right now because the doc seems old and the many changes I've tried don't
provoke any different behavior so I've not much to go on.  Frustrating.

Thanks in advance.

rw2

Re: trouble with expect_related

[Filip Sneppe (Cronos)]

Hi Rich,

On Sat, 2003-01-04 at 01:13, Rich Wellner wrote:
> I'm writing a module for ipfilters to deal with a protocol we use at work.
> You'll pardon me for being long winded, but I'm not positive what's going
> wrong.
> 
...
> 
> I claim it works because i see log messages and the helper is called as
> expected.
> 
> My problem is with the helper.  I parse the packet out and run
> ip_conntrack_expect_related but the incoming connection doesn't get passed
> through.  I have verified using tcpdump that the connection request does come
> from the remote machine, but my module doesn't seem to have registered the
> return right.  Anyway, here's the helper (more commentary follows code):
> 
...
>   ip_conntrack_expect_related(ct, &t, &mask, NULL);

FYI:
This is the "old" way of calling ip_conntrack_expect_related.
For quite a while now, there has been a newer infrastructure
for writing conntrack/nat helpers called "newnat". This
newer infrastructure solves some limitations of the old
code.

Since this newnat infrastructure was added to the mainline
kernel from early 2.4.20-pre releases onwards, and all the
netfilter conntrackers in CVS have been updated to this
infrastructure a long time ago, there really isn't much
interest in still using the old framework - all in all,
the newnat framework isn't that different if you're writing
a conntracker.

Have a look at the document describing newnat at:
http://cvs.netfilter.org/cgi-bin/cvsweb/netfilter/documentation/newnat-summary.txt
(or check out netfilter CVS of course)
and check out some conntrackers to learn about the
new API:
http://cvs.netfilter.org/cgi-bin/cvsweb/netfilter/patch-o-matic/extra/

The newnat patch that was included in the mainline kernel
for the 2.4.20 release and the patch can be found here:
http://cvs.netfilter.org/cgi-bin/cvsweb/netfilter/patch-o-matic/submitted/z-newnat16.patch
Since you've already studied the ftp conntracking code,
you can see how ip_conntrack_expect_related is called a
little differently in ip_conntrack_ftp.c in the newnat 
infrastructure.

The documentation you based your work on has, as you
may have guessed, not been updated to represent these
changes yet.

However, the oldnat stuff you based your conntracker 
on should be able to add your expectations without
any problems, so your problem will probably not get
solved by moving to the newnat API.

...
> 
> Anyone capable of helping me will probably recognize the general structure of
> the above code as being that from the existing ftp/irc modules.  I tried to
> use the doc on netfilter.org, but it seems to be pretty outdated, making, for
> example, references to structure elements that don't exist anymore.
> 
> Significantly the last printk above shows what I expect to see: 
> Jan  3 17:45:54 oxygen kernel: expect_related 131.225.80.34:0-207.252.250.2:38344
> 
> but the incoming connection, as i say above, never makes it through.
> 
> I've tried several different variations, but would be willing to try more if
> people have only suggestions but no answers.  I'm just a bit out of my wits
> right now because the doc seems old and the many changes I've tried don't
> provoke any different behavior so I've not much to go on.  Frustrating.
> 

Are you also letting RELATED connections in ? Most often,
people will use

/usr/sbin/iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j
ACCEPT

It would be a "bang-your-head-against-the-wall" type of
mistake not to let RELATED traffic trough when writing a
conntracker.

Other than that, does 

cat /proc/net/ip_conntrack | grep ^EXPECTING

turn up anything ? It should if the expectation
is added and the firewall hasn't seen this traffic yet.

Regards,
Filip

Re: trouble with expect_related

[Rich Wellner]

"Filip Sneppe (Cronos)" <filip.sneppe@cronos.be> writes:

> However, the oldnat stuff you based your conntracker 
> on should be able to add your expectations without
> any problems, so your problem will probably not get
> solved by moving to the newnat API.

Right, makes sense.  I can't see what's wrong with my code, seems like it
should just work.  Unfortunately, switching kernel isn't an option, I have to
support the 2.4 series with my module, not just the most current stuff.  

Can anyone else comment?

rw2

Re: trouble with expect_related

[Filip Sneppe]

On Mon, 2003-01-06 at 16:58, Rich Wellner wrote:
> "Filip Sneppe (Cronos)" <filip.sneppe@cronos.be> writes:
> 
> > However, the oldnat stuff you based your conntracker 
> > on should be able to add your expectations without
> > any problems, so your problem will probably not get
> > solved by moving to the newnat API.
> 
> Right, makes sense.  I can't see what's wrong with my code, seems like it
> should just work.  Unfortunately, switching kernel isn't an option, I have to
> support the 2.4 series with my module, not just the most current stuff.  

No problem.
 
> Can anyone else comment?
> 
Do you see the expected connection in /proc/net/ip_conntrack? It should
be preceded by "EXPECTING:" if it hasn't seen any traffic yet.

Regards,
Filip

Re: trouble with expect_related

[Rich Wellner]

Filip Sneppe <filip.sneppe@cronos.be> writes:

> On Mon, 2003-01-06 at 16:58, Rich Wellner wrote:
>> "Filip Sneppe (Cronos)" <filip.sneppe@cronos.be> writes:
>>
>> Can anyone else comment?
>> 
> Do you see the expected connection in /proc/net/ip_conntrack? It should
> be preceded by "EXPECTING:" if it hasn't seen any traffic yet.

Ok, this is very helpful.  I didn't know about that vfile.  There is the line
I expect there.

EXPECTING: proto=6 src=131.225.80.34 dst=207.252.250.2 sport=0 dport=41209

I don't know where to look to find out what proto 6 is, but the rest *seems*
ok.

The src, 131.225.80.34 is the machine I expect to be calling me back.  The
sport of 0 should be ok because I don't have that byte masked in my call to
expect_related the dport is correct.

I wasn't sure if the dst should be the NAT node or the private net node, so
I've tried it both ways:

EXPECTING: proto=6 src=131.225.80.34 dst=192.168.1.2 sport=0 dport=41330

And neither forwarded the connection.  Judging by the FTP example the dst
should be the private node, right?

And I reconfirmed using tcpdump that the incoming connection is still being
attempted and is on the correct port.

rw2

Re: trouble with expect_related

[Filip Sneppe]

On Tue, 2003-01-07 at 16:59, Rich Wellner wrote:
> Filip Sneppe <filip.sneppe@cronos.be> writes:
> > Do you see the expected connection in /proc/net/ip_conntrack? It should
> > be preceded by "EXPECTING:" if it hasn't seen any traffic yet.
> 
> Ok, this is very helpful.  I didn't know about that vfile.  There is the line
> I expect there.
> 
> EXPECTING: proto=6 src=131.225.80.34 dst=207.252.250.2 sport=0 dport=41209
> 
> I don't know where to look to find out what proto 6 is, but the rest *seems*
> ok. 

proto=6 means it's a tcp packet. /etc/protocols has all the
protocol numbers.

> The src, 131.225.80.34 is the machine I expect to be calling me back.  The
> sport of 0 should be ok because I don't have that byte masked in my call to
> expect_related the dport is correct.
> 
> I wasn't sure if the dst should be the NAT node or the private net node, so
> I've tried it both ways:
> 
> EXPECTING: proto=6 src=131.225.80.34 dst=192.168.1.2 sport=0 dport=41330
> 
> And neither forwarded the connection.  Judging by the FTP example the dst
> should be the private node, right?

I *think* that EXPECTING should show the packet as it appears on the
wire, since you should call ip_conntrack_change_expect in the nat helper
code to change the expectation. Not 100% sure though. You could try 
without NAT.

> And I reconfirmed using tcpdump that the incoming connection is still being
> attempted and is on the correct port.
>
As I pointed this out in my very first reply, and I suspect that this
may be the cause of the problem:

Are you letting the RELATED traffic in with the appropriate iptables
rules ? Can you post your firewall script, chain policies etc ?

I have never seen any reports of the connection tracking code not
picking up related TCP packets unless they were being blocked by
policy.

Regards,
Filip

Re: trouble with expect_related

[Filip Sneppe]

Rich,

I am just cc-ing this excerpt of our private mail exchange
about this problem to netfilter-devel again so it
gets archived for future developers to benefit from.

The reason for the trouble with ip_conntrack_expect_related
is that NAT setups require a ip_xxx_nat.c module.

Regards & good luck,
Filip

On Tue, 2003-01-07 at 21:41, Rich Wellner wrote:
> Filip Sneppe <filip.sneppe@cronos.be> writes:
> >
> > Is it possible for you to test things without NAT ? Might be
difficult if
> > you're doing this on a home MASQUERADING box with Internet
connectivity.
> >
> > If not, you're going to have to show more code, namely what you're
doing in
> > ip_xxx_nat.c.
> 
> Ugh, this is a train wreck.  I didn't understand that I needed both a
> ip_conntrack_dcap module and a ip_nat_dcap.c module.
> 
> I'll have to adapt one from the IRC I guess.
> 
> Thanks for the help.
> 
> rw2