working with libnetfilter_queue and linbetfilter_contrack

working with libnetfilter_queue and linbetfilter_contrack

[Mojtaba]

Hello Everyone,
I am working for a while on two projects (libnetfilter_queue and
linbetfilter_contrack) to get the decision of destined of packets that
arrived in our project. It greats to get the control of all packets.
But I confused a little.
In my solution i just want to forward all packets that are in the same
conditions (for example: all packets are received from specific
IP:PORT address) to another destination. I could add simply add new
rule in llinbetfilter_contrack list (like the samples that are exist
in linbetfilter_contrack/utility project).
But actually i want to use NFQUEUE to get all packets in my user-space
and then add new rule in linbetfilter_contrack list. In other words,
the verdict in my sulotions is not ACCEPT or DROP the packet, it
should add new rule in linbetfilter_contrack list if it is not exist.
Is it possible?
I am thinking about this, But  I am not sure it is correct or not?
For example:

static int cb(struct nfq_q_handle *qh, struct nfgenmsg *nfmsg,
         struct nfq_data *nfa, void *data)
{
   uint32_t id = print_pkt(nfa);
   printf("entering callback\n");
if (not exist in list){
ct = nfct_new();
   if (ct == NULL) {
       perror("nfct_new");
   return 0;
  }
Add_to_list();
}
return;
}



-- 
--Mojtaba Esfandiari.S

Re: working with libnetfilter_queue and linbetfilter_contrack

[Mojtaba]

Hello Everyone,
Any idea?

On Mon, Jun 17, 2019 at 2:12 PM Mojtaba <mespio@gmail.com> wrote:
>
> Hello Everyone,
> I am working for a while on two projects (libnetfilter_queue and
> linbetfilter_contrack) to get the decision of destined of packets that
> arrived in our project. It greats to get the control of all packets.
> But I confused a little.
> In my solution i just want to forward all packets that are in the same
> conditions (for example: all packets are received from specific
> IP:PORT address) to another destination. I could add simply add new
> rule in llinbetfilter_contrack list (like the samples that are exist
> in linbetfilter_contrack/utility project).
> But actually i want to use NFQUEUE to get all packets in my user-space
> and then add new rule in linbetfilter_contrack list. In other words,
> the verdict in my sulotions is not ACCEPT or DROP the packet, it
> should add new rule in linbetfilter_contrack list if it is not exist.
> Is it possible?
> I am thinking about this, But  I am not sure it is correct or not?
> For example:
>
> static int cb(struct nfq_q_handle *qh, struct nfgenmsg *nfmsg,
>          struct nfq_data *nfa, void *data)
> {
>    uint32_t id = print_pkt(nfa);
>    printf("entering callback\n");
> if (not exist in list){
> ct = nfct_new();
>    if (ct == NULL) {
>        perror("nfct_new");
>    return 0;
>   }
> Add_to_list();
> }
> return;
> }
>
>
>
> --
> --Mojtaba Esfandiari.S



-- 
--Mojtaba Esfandiari.S

Re: working with libnetfilter_queue and linbetfilter_contrack

[Florian Westphal]

Mojtaba <mespio@gmail.com> wrote:
> I am working for a while on two projects (libnetfilter_queue and
> linbetfilter_contrack) to get the decision of destined of packets that
> arrived in our project. It greats to get the control of all packets.
> But I confused a little.
> In my solution i just want to forward all packets that are in the same
> conditions (for example: all packets are received from specific
> IP:PORT address) to another destination. I could add simply add new
> rule in llinbetfilter_contrack list (like the samples that are exist
> in linbetfilter_contrack/utility project).
> But actually i want to use NFQUEUE to get all packets in my user-space
> and then add new rule in linbetfilter_contrack list. In other words,
> the verdict in my sulotions is not ACCEPT or DROP the packet, it
> should add new rule in linbetfilter_contrack list if it is not exist.
> Is it possible?

Yes, but that doesn't make sense because the kernel will add a conntrack
entry itself if no entry existed.
Or are you dropping packets in NEW state?
Or are you talking about conntrack expectations?

A conntrack entry itself doesn't accept or forward a packet.

It just means that next packet of same flow will find the entry and
rules like iptables ... -m conntrack --ctstate NEW/ESTABLISHED etc.
will match.

Re: working with libnetfilter_queue and linbetfilter_contrack

[Mojtaba]

Yes, For this reason, i should add conntrack entry before the kernel
do in my userspace project. Because i have to forward the packet to
another destination, i used --src-nat and --dst-nat options while
adding new  conntrack entry. Just like as obvious in below code:
nfct_set_attr_u8(ct, ATTR_L3PROTO, AF_INET);
nfct_set_attr_u32(ct, ATTR_IPV4_SRC, inet_addr("192.168.133.140"));
nfct_set_attr_u32(ct, ATTR_IPV4_DST, inet_addr("192.168.133.108"));
nfct_set_attr_u8(ct, ATTR_L4PROTO, IPPROTO_UDP);
nfct_set_attr_u16(ct, ATTR_PORT_SRC, htons(6000));
nfct_set_attr_u16(ct, ATTR_PORT_DST, htons(5005));
nfct_setobjopt(ct, NFCT_SOPT_SETUP_REPLY);
nfct_set_attr_u32(ct, ATTR_TIMEOUT, 60);
nfct_set_attr_u32(ct, ATTR_SNAT_IPV4, inet_addr("192.168.133.108"));
nfct_set_attr_u32(ct, ATTR_DNAT_IPV4, inet_addr("192.168.133.150"));
nfct_set_attr_u16(ct, ATTR_SNAT_PORT, htons(5070));
nfct_set_attr_u16(ct, ATTR_DNAT_PORT, htons(6000));

As far as i know, it is possible to delegate verdict of packets to
user-space, Here is the main point that is deriving me confused.
Suppose i used this rule in IPTABLE:
iptables -A INPUT -p udp --dport 5005  -j NQUEUE --queue-num 0
Then how we could make verdict to forward the packet to another
destination?  Do i could implement my solution in this way or i have
to use libnetfilter_contrack like as above sample code?
WIth Best Regards.Mojtaba


On Tue, Jun 18, 2019 at 2:20 PM Florian Westphal <fw@strlen.de> wrote:
>
> Mojtaba <mespio@gmail.com> wrote:
> > I am working for a while on two projects (libnetfilter_queue and
> > linbetfilter_contrack) to get the decision of destined of packets that
> > arrived in our project. It greats to get the control of all packets.
> > But I confused a little.
> > In my solution i just want to forward all packets that are in the same
> > conditions (for example: all packets are received from specific
> > IP:PORT address) to another destination. I could add simply add new
> > rule in llinbetfilter_contrack list (like the samples that are exist
> > in linbetfilter_contrack/utility project).
> > But actually i want to use NFQUEUE to get all packets in my user-space
> > and then add new rule in linbetfilter_contrack list. In other words,
> > the verdict in my sulotions is not ACCEPT or DROP the packet, it
> > should add new rule in linbetfilter_contrack list if it is not exist.
> > Is it possible?
>
> Yes, but that doesn't make sense because the kernel will add a conntrack
> entry itself if no entry existed.
> Or are you dropping packets in NEW state?
> Or are you talking about conntrack expectations?
>
> A conntrack entry itself doesn't accept or forward a packet.
>
> It just means that next packet of same flow will find the entry and
> rules like iptables ... -m conntrack --ctstate NEW/ESTABLISHED etc.
> will match.



-- 
--Mojtaba Esfandiari.S

Re: working with libnetfilter_queue and linbetfilter_contrack

[Florian Westphal]

Mojtaba <mespio@gmail.com> wrote:
> Yes, For this reason, i should add conntrack entry before the kernel do in
> my userspace project. Because i have to forward the packet to another
> destination, i used --src-nat and --dst-nat options while adding new
>  conntrack entry. Just like as obvious in below code:
> nfct_set_attr_u8(ct, ATTR_L3PROTO, AF_INET);
> nfct_set_attr_u32(ct, ATTR_IPV4_SRC, inet_addr("192.168.133.140"));
> nfct_set_attr_u32(ct, ATTR_IPV4_DST, inet_addr("192.168.133.108"));
> nfct_set_attr_u8(ct, ATTR_L4PROTO, IPPROTO_UDP);
> nfct_set_attr_u16(ct, ATTR_PORT_SRC, htons(6000));
> nfct_set_attr_u16(ct, ATTR_PORT_DST, htons(5005));
> nfct_setobjopt(ct, NFCT_SOPT_SETUP_REPLY);
> nfct_set_attr_u32(ct, ATTR_TIMEOUT, 60);
> 
> *nfct_set_attr_u32(ct, ATTR_SNAT_IPV4,
> inet_addr("192.168.133.108"));nfct_set_attr_u32(ct, ATTR_DNAT_IPV4,
> inet_addr("192.168.133.150"));nfct_set_attr_u16(ct, ATTR_SNAT_PORT,
> htons(5070));*
> 
> *nfct_set_attr_u16(ct, ATTR_DNAT_PORT, htons(6000));*
> 
> As far as i know, it is possible to delegate verdict of packets to
> user-space, Here is the main point that is deriving me confused. Suppose i
> used this rule in IPTABLE:
> iptables -A INPUT -p udp --dport 5005  -j NQUEUE --queue-num 0
> Then how we could make verdict to forward the packet to another
> destination?

You can't, INPUT is too late and NFQUEUE can't tell kernel to do nat.

You could do what you want by placing NFQUEUE in raw PREROUTING,
but in that case all packets would get queued to userspace because
no conntrack information is available yet.

But if you create the conntrack entry, then after accept verdict the
kernel would find the conntrack entry in place and perform nat for it.

It would be possible to extend nfnetlink_queue to also allow changing
NAT properties of a conntrack entry provided the conntrack has not been
confirmed yet but it would require kernel changes.

So, best option afaics is to use libnetfilter_conntrack to insert
a new conntrack entry from the nfq callback.

Re: working with libnetfilter_queue and linbetfilter_contrack

[Florian Westphal]

Mojtaba <mespio@gmail.com> wrote:
> Thanks Florian so much.
> According the last paragraf of email i get the best way is i should use
> libnetfilter_conntrack to insert a new conntrack entry in my userspace that
> is called from raw PREROUTING table as NQUEUE callback queue.
> Is it right underestanding?

Yes, but since you did not exactly say what you're trying to do
there might be better ways (ipvs, nft maps, etc).

Nfqueue is slow.

Re: working with libnetfilter_queue and linbetfilter_contrack

[Mojtaba]

Thanks a lot.
Then let me describe what i am doing.
In VoIP networks, One of the ways to solve the one-way audio issue is
TURN. In this case both endpoint have to send their media (voice as
RTP) to server. In this conditions the server works as B2BUA. Because
of the server is processing the media (get media from one hand and
relay it to another hand), It usages a lot of resource of server. So I
am implementing  a new module to do this in kernel level. I test this
idea in my laboratory by adding conntrack entry manually in server and
all things works great. But i need to get more  idea to do this
project in best way and high performance, because the QoS very
importance in VoIP networks. What is the best way? Let me know more
about this.
With Best Regards.Mojtaba

On Tue, Jun 18, 2019 at 5:53 PM Florian Westphal <fw@strlen.de> wrote:
>
> Mojtaba <mespio@gmail.com> wrote:
> > Thanks Florian so much.
> > According the last paragraf of email i get the best way is i should use
> > libnetfilter_conntrack to insert a new conntrack entry in my userspace that
> > is called from raw PREROUTING table as NQUEUE callback queue.
> > Is it right underestanding?
>
> Yes, but since you did not exactly say what you're trying to do
> there might be better ways (ipvs, nft maps, etc).
>
> Nfqueue is slow.



-- 
--Mojtaba Esfandiari.S

Re: working with libnetfilter_queue and linbetfilter_contrack

[Florian Westphal]

Mojtaba <mespio@gmail.com> wrote:
> Then let me describe what i am doing.
> In VoIP networks, One of the ways to solve the one-way audio issue is
> TURN. In this case both endpoint have to send their media (voice as
> RTP) to server. In this conditions the server works as B2BUA. Because
> of the server is processing the media (get media from one hand and
> relay it to another hand), It usages a lot of resource of server. So I
> am implementing  a new module to do this in kernel level. I test this
> idea in my laboratory by adding conntrack entry manually in server and
> all things works great. But i need to get more  idea to do this
> project in best way and high performance, because the QoS very
> importance in VoIP networks. What is the best way? Let me know more
> about this.

In that case I wonder why you need nfqueue at all.

Isn't it enough for the proxy to inject a conntrack entry with the
expected endpoint addresses of the media stream?

I would expect that your proxy consumes/reads the sdp messages from
the client already, or are you doing that via nfqueue?

I would probably use tproxy+normal socket api for the signalling
packets and insert conntrack entries for the rtp/media streams
via libnetfilter_conntrack, this way, the media streams stay in kernel.

Re: working with libnetfilter_queue and linbetfilter_contrack

[Mojtaba]

Hello,
Absolutely of course, i used exactly the same way in my test-case. I
added 200 entry in libnetfilter_conntrack for 200 concurrent call. In
reality i have to extract the address of media stream for both
endpoints in SIP-Proxy server then send them to user-space project in
another machine over TCP connection. Here is what i do in test-case
project. I have to change conntrack_create_nat.c like below:

int i = 10000;
int end = 30000
int MAX_CALL = 200;
int j = 10000 + (MAX_CALL*4-4);
while(i<=j) {

   nfct_set_attr_u8(ct, ATTR_L3PROTO, AF_INET);
   nfct_set_attr_u32(ct, ATTR_IPV4_SRC, inet_addr("192.168.133.140"));
         //endpoint A
   nfct_set_attr_u32(ct, ATTR_IPV4_DST, inet_addr("192.168.133.108"));

   //nfct_set_attr_u8(ct, ATTR_L4PROTO, IPPROTO_TCP);
   nfct_set_attr_u8(ct, ATTR_L4PROTO, IPPROTO_UDP);
   nfct_set_attr_u16(ct, ATTR_PORT_SRC, htons(6000));
   nfct_set_attr_u16(ct, ATTR_PORT_DST, htons(i));

   nfct_setobjopt(ct, NFCT_SOPT_SETUP_REPLY);

   //nfct_set_attr_u8(ct, ATTR_TCP_STATE, TCP_CONNTRACK_SYN_SENT);
   nfct_set_attr_u32(ct, ATTR_TIMEOUT, 200);

   nfct_set_attr_u32(ct, ATTR_SNAT_IPV4, inet_addr("192.168.133.108"));
   nfct_set_attr_u32(ct, ATTR_DNAT_IPV4,
inet_addr("192.168.133.150"));               //endpoint B

        nfct_set_attr_u16(ct, ATTR_SNAT_PORT, htons(i+2));
        nfct_set_attr_u16(ct, ATTR_DNAT_PORT, htons(6000));

   ret = nfct_query(h, NFCT_Q_CREATE, ct);
   i+=4;
   printf("TEST: create conntrack ");
   if (ret == -1)
      printf("(%d)(%s)\n", ret, strerror(errno));
   else
      printf("(OK)\n");
}

But I have to add  a rule in IPTABLE to not add any conntrack entry by
kernel, because as soos as the callee answer the call(received 200OK
SIP MESSAGE), it will start to send it's media (RTP).In this regards
it would create conntrack entry sooner than user-space.
iptables -A INPUT -p udp --dport 10000:30000 -j DROP
Is it right table to deny adding any conntrack entry or not?
Anyway i appreciate your guide. I was in dilemma to used
libnetfilter_conntrack or libnetfilter_queue. Thanks
WIth Best Regards.Mojtaba



On Tue, Jun 18, 2019 at 6:30 PM Florian Westphal <fw@strlen.de> wrote:
>
> Mojtaba <mespio@gmail.com> wrote:
> > Then let me describe what i am doing.
> > In VoIP networks, One of the ways to solve the one-way audio issue is
> > TURN. In this case both endpoint have to send their media (voice as
> > RTP) to server. In this conditions the server works as B2BUA. Because
> > of the server is processing the media (get media from one hand and
> > relay it to another hand), It usages a lot of resource of server. So I
> > am implementing  a new module to do this in kernel level. I test this
> > idea in my laboratory by adding conntrack entry manually in server and
> > all things works great. But i need to get more  idea to do this
> > project in best way and high performance, because the QoS very
> > importance in VoIP networks. What is the best way? Let me know more
> > about this.
>
> In that case I wonder why you need nfqueue at all.
>
> Isn't it enough for the proxy to inject a conntrack entry with the
> expected endpoint addresses of the media stream?
>
> I would expect that your proxy consumes/reads the sdp messages from
> the client already, or are you doing that via nfqueue?
>
> I would probably use tproxy+normal socket api for the signalling
> packets and insert conntrack entries for the rtp/media streams
> via libnetfilter_conntrack, this way, the media streams stay in kernel.



-- 
--Mojtaba Esfandiari.S

Re: working with libnetfilter_queue and linbetfilter_contrack

[Mojtaba]

Hello,
I appreciate your guide again,
That works great. It would be a creative method to out of concern of
one-way issue in VoIP networks. The new module based on this method
would be released in Kamailio project as soon.
Keeping an eye out for it.
Thanks.
With regards.Mojtaba

On Wed, Jun 19, 2019 at 11:20 AM Mojtaba <mespio@gmail.com> wrote:
>
> Hello,
> Absolutely of course, i used exactly the same way in my test-case. I
> added 200 entry in libnetfilter_conntrack for 200 concurrent call. In
> reality i have to extract the address of media stream for both
> endpoints in SIP-Proxy server then send them to user-space project in
> another machine over TCP connection. Here is what i do in test-case
> project. I have to change conntrack_create_nat.c like below:
>
> int i = 10000;
> int end = 30000
> int MAX_CALL = 200;
> int j = 10000 + (MAX_CALL*4-4);
> while(i<=j) {
>
>    nfct_set_attr_u8(ct, ATTR_L3PROTO, AF_INET);
>    nfct_set_attr_u32(ct, ATTR_IPV4_SRC, inet_addr("192.168.133.140"));
>          //endpoint A
>    nfct_set_attr_u32(ct, ATTR_IPV4_DST, inet_addr("192.168.133.108"));
>
>    //nfct_set_attr_u8(ct, ATTR_L4PROTO, IPPROTO_TCP);
>    nfct_set_attr_u8(ct, ATTR_L4PROTO, IPPROTO_UDP);
>    nfct_set_attr_u16(ct, ATTR_PORT_SRC, htons(6000));
>    nfct_set_attr_u16(ct, ATTR_PORT_DST, htons(i));
>
>    nfct_setobjopt(ct, NFCT_SOPT_SETUP_REPLY);
>
>    //nfct_set_attr_u8(ct, ATTR_TCP_STATE, TCP_CONNTRACK_SYN_SENT);
>    nfct_set_attr_u32(ct, ATTR_TIMEOUT, 200);
>
>    nfct_set_attr_u32(ct, ATTR_SNAT_IPV4, inet_addr("192.168.133.108"));
>    nfct_set_attr_u32(ct, ATTR_DNAT_IPV4,
> inet_addr("192.168.133.150"));               //endpoint B
>
>         nfct_set_attr_u16(ct, ATTR_SNAT_PORT, htons(i+2));
>         nfct_set_attr_u16(ct, ATTR_DNAT_PORT, htons(6000));
>
>    ret = nfct_query(h, NFCT_Q_CREATE, ct);
>    i+=4;
>    printf("TEST: create conntrack ");
>    if (ret == -1)
>       printf("(%d)(%s)\n", ret, strerror(errno));
>    else
>       printf("(OK)\n");
> }
>
> But I have to add  a rule in IPTABLE to not add any conntrack entry by
> kernel, because as soos as the callee answer the call(received 200OK
> SIP MESSAGE), it will start to send it's media (RTP).In this regards
> it would create conntrack entry sooner than user-space.
> iptables -A INPUT -p udp --dport 10000:30000 -j DROP
> Is it right table to deny adding any conntrack entry or not?
> Anyway i appreciate your guide. I was in dilemma to used
> libnetfilter_conntrack or libnetfilter_queue. Thanks
> WIth Best Regards.Mojtaba
>
>
>
> On Tue, Jun 18, 2019 at 6:30 PM Florian Westphal <fw@strlen.de> wrote:
> >
> > Mojtaba <mespio@gmail.com> wrote:
> > > Then let me describe what i am doing.
> > > In VoIP networks, One of the ways to solve the one-way audio issue is
> > > TURN. In this case both endpoint have to send their media (voice as
> > > RTP) to server. In this conditions the server works as B2BUA. Because
> > > of the server is processing the media (get media from one hand and
> > > relay it to another hand), It usages a lot of resource of server. So I
> > > am implementing  a new module to do this in kernel level. I test this
> > > idea in my laboratory by adding conntrack entry manually in server and
> > > all things works great. But i need to get more  idea to do this
> > > project in best way and high performance, because the QoS very
> > > importance in VoIP networks. What is the best way? Let me know more
> > > about this.
> >
> > In that case I wonder why you need nfqueue at all.
> >
> > Isn't it enough for the proxy to inject a conntrack entry with the
> > expected endpoint addresses of the media stream?
> >
> > I would expect that your proxy consumes/reads the sdp messages from
> > the client already, or are you doing that via nfqueue?
> >
> > I would probably use tproxy+normal socket api for the signalling
> > packets and insert conntrack entries for the rtp/media streams
> > via libnetfilter_conntrack, this way, the media streams stay in kernel.
>
>
>
> --
> --Mojtaba Esfandiari.S



-- 
--Mojtaba Esfandiari.S