netfilter_queue : use --queue-balance in multithreaded environment

netfilter_queue : use --queue-balance in multithreaded environment

[Arif Hossain]

[-- Attachment #1: Type: text/plain, Size: 1310 bytes --]

Hi,

I'm using libnetfilter_queue for userspace modification of
incoming/outgoing packets. I have been using a single threaded model.
But i've found out that from 2.6.31 kernel, its possible to have
different queues for different connection. so i was wandering if its
possible to manage each queues in different threads.

normally i set up the queue handling like bellow:

struct nfq_handle * h = nfq_open();
nfq_unbind_pf(h, AF_NET);
nfq_bind_pf(h,m AF_NET);
struct nfq_q_handle(h, 0, &cb, NULL);
nfq_set_mode(qh, NFQNL_COPY_PACKET, 0xffff);

now if i want to manage like 100 queues, i will pack h,qh and que_num in
a structure and loop over it to initialize.

now my question is :

if i initialize above in the main thread and want to run the callbacks
in threads, is it enough to run infinite loop in a function which will
be given to pthread_create()? Will it run the callbacks in threads? 

i'm not sure, but my understanding tells me, a packet is popped from the
queues when nfq_set_verdict is returned. so i need to run
nfq_set_verdict in separate threads so that i packets can be popped from
queues parallel. 
-- 
"You have a voice"
                                        ----The King's Speech

Public Key : $ gpg --keyserver keyserver.ubuntu.com --recv-key C88CFC23

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: netfilter_queue : use --queue-balance in multithreaded environment

[Florian Westphal]

Arif Hossain <freefall1986@gmail.com> wrote:
> I'm using libnetfilter_queue for userspace modification of
> incoming/outgoing packets. I have been using a single threaded model.
> But i've found out that from 2.6.31 kernel, its possible to have
> different queues for different connection. so i was wandering if its
> possible to manage each queues in different threads.

Its usually simpler to just run several instances of the same program,
easier to maintain/debug and avoids locking/race conditions...

If you have requirements that make this a bad choice (e.g. memory
usage), then yes, you can use threads.

See
http://git.breakpoint.cc/cgi-bin/gitweb.cgi?p=fw/afcd.git;a=blob;f=src/in/in_nl.c;h=e298bd073f3c7e0c62432510c2b3f7de953df1ec;hb=74a2df00cdee5f3ff8360b32d711d5db086b5355#l351

for a program that does this.

> if i initialize above in the main thread and want to run the callbacks
> in threads, is it enough to run infinite loop in a function which will
> be given to pthread_create()? Will it run the callbacks in threads? 

Yes.

> i'm not sure, but my understanding tells me, a packet is popped from the
> queues when nfq_set_verdict is returned. so i need to run
> nfq_set_verdict in separate threads so that i packets can be popped from
> queues parallel.

I don't understand this part.  Basically all the processing do this
(see url above for full example):

 static int nfnl_queue_cb(struct nfq_q_handle *qh, struct nfgenmsg *nfmsg,
               struct nfq_data *nfa, void *data) {
        [..]
        ret = nfq_get_payload(nfa, &pkt);
	do_something(pkt);
        return nfq_set_verdict2( .. )
}

static void *_thread_main(void *arg) {
	[..]
        while (!thread_should_stop) {
                ssize_t rv = read(fd, buf, nfq_buf_len);
                if (rv > 0)
                        nfq_handle_packet(nfq_h, buf, (int) rv);
                else
                        /* error ... */
        }
	/* cleanup */
        return NULL;
}

All the threads then run nfnl_queue_cb() independently.
Of course, you'll need to add proper locking if they need to
access shared data structures...