Re: NFQUEUE the plot is growing...

[Alessandro Vesely <vesely@tana.it>]

[-- Attachment #1: Type: text/plain, Size: 1720 bytes --]

On 12/May/11 11:40, nowhere wrote:
>> It seems enough to avoid delaying the call to nfq_set_verdict for the
>> first packet of a burst.  For a shot in the dark, packets seem to get
>> lost if they arrive between the first one and the corresponding call
>> to nfq_set_verdict.  Indeed, setting a fixed real_delay of 0.2, with
>> ping -i 0.2 it looses no packets, with ping -i 0.19 it looses just the
>> second one, with ping -i 0.09 icmp_reqs #2 and #3.
>> 
>> No error is returned, whether NETLINK_NO_ENOBUFS is set or not.
> 
> Well, seems like this is the case. If nfqueue becomes empty, first
> enqueued packet must not be delayed.

I retract, possibly I've been too hasty blaming nfnetlink queue.  I
made a simple variation of nfqnl_test.c --which I attach.  It just
accepts the previous packet id.  The "last" packet is obviously always
lost.  Because of this bug(?), I also loose the second packet of a
sequence of pings, no matter the speed.

However, if I "ping -c 1" using two terminal windows, I correctly
receive all odd ids in one window and even ones in the other (except
last pkt).  In this case, I delay every packet.  Also, if I run a
sequence from a window, and, immediately after it starts, run a single
ping using the other window, then both the single ping and the
sequence (except last pkt) go correctly through.

I don't understand how come the kernel+filter system can distinguish
between a second packet coming as part of a sequence and a second
packet coming asynchronously, given that packets are not inspected.
Nice puzzle, isn't it?


NB, I used iptables -t mangle -A POSTROUTING -p icmp -d 172.25.197.158
-j NFQUEUE --queue-num 13, as in
http://www.spinics.net/lists/netfilter/msg50829.html

[-- Attachment #2: main2.c --]
[-- Type: text/plain, Size: 3505 bytes --]

/*

$ ping -c 8 172.25.197.158
PING 172.25.197.158 (172.25.197.158) 56(84) bytes of data.
64 bytes from 172.25.197.158: icmp_req=1 ttl=128 time=1010 ms
64 bytes from 172.25.197.158: icmp_req=3 ttl=128 time=1000 ms
64 bytes from 172.25.197.158: icmp_req=4 ttl=128 time=1000 ms
64 bytes from 172.25.197.158: icmp_req=5 ttl=128 time=999 ms
64 bytes from 172.25.197.158: icmp_req=6 ttl=128 time=1000 ms
64 bytes from 172.25.197.158: icmp_req=7 ttl=128 time=1010 ms

--- 172.25.197.158 ping statistics ---
8 packets transmitted, 6 received, 25% packet loss, time 7020ms
rtt min/avg/max/mdev = 999.452/1003.574/1010.633/4.883 ms, pipe 2


# grep -E ^ *13 /proc/net/netfilter/nfnetlink_queue
   13  13584     1 1     0     0     0        8  1

   ^queue_num,
       ^peer_pid,
                 ^queue_total,
                   ^copy_mode,
                          ^copy_range,
                               ^queue_dropped,
                                      ^queue_user_dropped,
                                              ^id_sequence,
                                                  ^1
*/



#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <netinet/in.h>
#include <linux/types.h>
#include <linux/netfilter.h>		/* for NF_ACCEPT */

#include <libnetfilter_queue/libnetfilter_queue.h>

#include <errno.h>

struct packet_data
{
	u_int32_t id, good;
};

static int cb(struct nfq_q_handle *qh, struct nfgenmsg *nu,
	      struct nfq_data *nfa, void *data)
{
	struct packet_data *pd = (struct packet_data*)data;
	struct nfqnl_msg_packet_hdr *ph = nfq_get_msg_packet_hdr(nfa);
	
	if (pd && ph)
	{
		if (pd->good) // since last time
			nfq_set_verdict(qh, pd->id, NF_ACCEPT, 0, NULL);

		pd->id = ntohl(ph->packet_id);
		pd->good = 1;
		printf("received packet %d\n", pd->id);
	}
	
	return 0;

	(void)nu;
}

int main()
{
	struct nfq_handle *h;
	struct nfq_q_handle *qh;
	struct packet_data pd;
	int fd;
	int rv;
	char buf[4096] __attribute__ ((aligned));

	printf("opening library handle\n");
	h = nfq_open();
	if (!h) {
		fprintf(stderr, "error during nfq_open()\n");
		exit(1);
	}

	printf("unbinding existing nf_queue handler for AF_INET (if any)\n");
	if (nfq_unbind_pf(h, AF_INET) < 0) {
		fprintf(stderr, "error during nfq_unbind_pf()\n");
		exit(1);
	}

	printf("binding nfnetlink_queue as nf_queue handler for AF_INET\n");
	if (nfq_bind_pf(h, AF_INET) < 0) {
		fprintf(stderr, "error during nfq_bind_pf()\n");
		exit(1);
	}

	pd.good = 0;
	qh = nfq_create_queue(h,  13 /* <--queue number here */, &cb, &pd);
	if (!qh) {
		fprintf(stderr, "error during nfq_create_queue()\n");
		exit(1);
	}

	if (nfq_set_mode(qh, NFQNL_COPY_META, 0) < 0) {
		fprintf(stderr, "can't set packet_copy mode\n");
		exit(1);
	}

	fd = nfq_fd(h);

	for (;;) {
		if ((rv = recv(fd, buf, sizeof(buf), 0)) >= 0) {
			nfq_handle_packet(h, buf, rv);
			continue;
		}
		/* if the computer is slower than the network the buffer
		* may fill up. Depending on the application, this error
		* may be ignored */		
		if (errno == ENOBUFS) {
			printf("pkt lost!!\n");
			continue;
		}
		printf("recv failed: errno=%d (%s)\n",
			errno, strerror(errno));
	}

	printf("unbinding from queue 0\n");
	nfq_destroy_queue(qh);

#ifdef INSANE
	/* normally, applications SHOULD NOT issue this command, since
	 * it detaches other programs/sockets from AF_INET, too ! */
	printf("unbinding from AF_INET\n");
	nfq_unbind_pf(h, AF_INET);
#endif

	printf("closing library handle\n");
	nfq_close(h);

	exit(0);
}