Hook location of IMQ

Hook location of IMQ

[Steve (Telsat Broadband)]

Hi All,

I've posted a couple of questions over on linuximq.net but the discussion
there seems quiet, so I'll try here to see if anyone here can point me in
the right direction.

Currently I use IMQ devices and TC to limit bandwidth to clients; this is
all working very well, except that the byte counters I'm relying on for
counting the clients data seems to be 'before' IMQ does its work.

For example; I've got rules in the 'mangle/forward' table for assigning the
clients data to the IMQ device and rules in the 'filter/forward' table which
matches the client's data and I'm counting their traffic from here.  

However, according to this packet flow show on linuximq.net
(http://www.docum.org/docum.org/kptd/) the IMQ hook is after 'POSTROUTING'
which means that even though I'm using '-j IMQ' in the 'mangle/forward'
table to limit the bandwidth before counting; the counters are still
counting all packets; including dropped ones by IMQ.

There doesn't seem to be any more 'chains' after the IMQ hook which I could
rely upon to 'count' the data after IMQ has done its job.

I realise that when compiling the kernel, I can choose where IMQ hooks in
(before or after NAT); currently I have selected as 'AB'.

What I'd like to know is; 

a) Is there something I'm missing; is there somewhere I can count the
packets after IMQ's work is done?
b) If not, is there some way I can modify the IMQ hook to be in-between the
'mangle/forward' and 'filter/forward' chains.

Any help/comments are greatly appreciated.

Thanks
Steve.

Re: Hook location of IMQ

[GGounot]

Hi.

Did you try IFB instead of IMQ ?

"The Intermediate Functional Block device is the successor to the IMQ 
iptables module that was never integrated."
http://www.linuxfoundation.org/collaborate/workgroups/networking/ifb


Le 17/09/2014 01:15, Steve (Telsat Broadband) a écrit :
> Hi All,
>
> I've posted a couple of questions over on linuximq.net but the discussion
> there seems quiet, so I'll try here to see if anyone here can point me in
> the right direction.
>
> Currently I use IMQ devices and TC to limit bandwidth to clients; this is
> all working very well, except that the byte counters I'm relying on for
> counting the clients data seems to be 'before' IMQ does its work.
>
> For example; I've got rules in the 'mangle/forward' table for assigning the
> clients data to the IMQ device and rules in the 'filter/forward' table which
> matches the client's data and I'm counting their traffic from here.
>
> However, according to this packet flow show on linuximq.net
> (http://www.docum.org/docum.org/kptd/) the IMQ hook is after 'POSTROUTING'
> which means that even though I'm using '-j IMQ' in the 'mangle/forward'
> table to limit the bandwidth before counting; the counters are still
> counting all packets; including dropped ones by IMQ.
>
> There doesn't seem to be any more 'chains' after the IMQ hook which I could
> rely upon to 'count' the data after IMQ has done its job.
>
> I realise that when compiling the kernel, I can choose where IMQ hooks in
> (before or after NAT); currently I have selected as 'AB'.
>
> What I'd like to know is;
>
> a) Is there something I'm missing; is there somewhere I can count the
> packets after IMQ's work is done?
> b) If not, is there some way I can modify the IMQ hook to be in-between the
> 'mangle/forward' and 'filter/forward' chains.
>
> Any help/comments are greatly appreciated.
>
> Thanks
> Steve.
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe lartc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

RE: Hook location of IMQ

[Steve (Telsat Broadband)]

Hi GGounot,

No, to be honest, I'd never even heard of IFB.  I'm reviewing all the info
now.

Thanks very much for your reply.

Thanks
Steve




-----Original Message-----
From: GGounot [mailto:g.gounot@laposte.net] 
Sent: Wednesday, 17 September 2014 6:10 PM
To: Steve (Telsat Broadband); lartc@vger.kernel.org
Subject: Re: Hook location of IMQ

Hi.

Did you try IFB instead of IMQ ?

"The Intermediate Functional Block device is the successor to the IMQ
iptables module that was never integrated."
http://www.linuxfoundation.org/collaborate/workgroups/networking/ifb


Le 17/09/2014 01:15, Steve (Telsat Broadband) a écrit :
> Hi All,
>
> I've posted a couple of questions over on linuximq.net but the discussion
> there seems quiet, so I'll try here to see if anyone here can point me in
> the right direction.
>
> Currently I use IMQ devices and TC to limit bandwidth to clients; this is
> all working very well, except that the byte counters I'm relying on for
> counting the clients data seems to be 'before' IMQ does its work.
>
> For example; I've got rules in the 'mangle/forward' table for assigning
the
> clients data to the IMQ device and rules in the 'filter/forward' table
which
> matches the client's data and I'm counting their traffic from here.
>
> However, according to this packet flow show on linuximq.net
> (http://www.docum.org/docum.org/kptd/) the IMQ hook is after 'POSTROUTING'
> which means that even though I'm using '-j IMQ' in the 'mangle/forward'
> table to limit the bandwidth before counting; the counters are still
> counting all packets; including dropped ones by IMQ.
>
> There doesn't seem to be any more 'chains' after the IMQ hook which I
could
> rely upon to 'count' the data after IMQ has done its job.
>
> I realise that when compiling the kernel, I can choose where IMQ hooks in
> (before or after NAT); currently I have selected as 'AB'.
>
> What I'd like to know is;
>
> a) Is there something I'm missing; is there somewhere I can count the
> packets after IMQ's work is done?
> b) If not, is there some way I can modify the IMQ hook to be in-between
the
> 'mangle/forward' and 'filter/forward' chains.
>
> Any help/comments are greatly appreciated.
>
> Thanks
> Steve.
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe lartc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

RE: Hook location of IMQ

[Steve (Telsat Broadband)]

Hi All/GGounot,

I've had a good review of the IFB, but it doesn't seem to have very good
documentation on its usage/implementation (that I've found anyway). 

IMQ has worked very well for my purpose, but the only issue I have is where
it is hooking.  I need a place (after PRE-ROUTING NAT) to be able to mark
packets and then count the ones successfully delivered after they've passed
through IMQ.

The best place I could find would be to have IMQ hook in 'before' the mangle
table in POSTROUTING.  

I'm not that familiar with NF hooks, but would it be possible to modify this
in some way to have IMQ hook in before the mangle table in PostRouting?

 /* imq_egress_ipv4 */
 .hook = imq_nf_hook,
 .owner = THIS_MODULE,
 .pf = PF_INET,
 .hooknum = NF_INET_POST_ROUTING,
#if defined(CONFIG_IMQ_BEHAVIOR_AA) || defined(CONFIG_IMQ_BEHAVIOR_BA)
 .priority = NF_IP_PRI_LAST,
#else
 .priority = NF_IP_PRI_NAT_SRC - 1,
#endif
 },


Thanks.
Steve.



-----Original Message-----
From: Steve (Telsat Broadband) [mailto:steve@telsatbb.vu] 
Sent: Wednesday, 17 September 2014 8:43 PM
To: 'GGounot'; 'lartc@vger.kernel.org'
Subject: RE: Hook location of IMQ

Hi GGounot,

No, to be honest, I'd never even heard of IFB.  I'm reviewing all the info
now.

Thanks very much for your reply.

Thanks
Steve




-----Original Message-----
From: GGounot [mailto:g.gounot@laposte.net]
Sent: Wednesday, 17 September 2014 6:10 PM
To: Steve (Telsat Broadband); lartc@vger.kernel.org
Subject: Re: Hook location of IMQ

Hi.

Did you try IFB instead of IMQ ?

"The Intermediate Functional Block device is the successor to the IMQ
iptables module that was never integrated."
http://www.linuxfoundation.org/collaborate/workgroups/networking/ifb


Le 17/09/2014 01:15, Steve (Telsat Broadband) a écrit :
> Hi All,
>
> I've posted a couple of questions over on linuximq.net but the 
> discussion there seems quiet, so I'll try here to see if anyone here 
> can point me in the right direction.
>
> Currently I use IMQ devices and TC to limit bandwidth to clients; this 
> is all working very well, except that the byte counters I'm relying on 
> for counting the clients data seems to be 'before' IMQ does its work.
>
> For example; I've got rules in the 'mangle/forward' table for 
> assigning the clients data to the IMQ device and rules in the 
> 'filter/forward' table which matches the client's data and I'm counting
their traffic from here.
>
> However, according to this packet flow show on linuximq.net
> (http://www.docum.org/docum.org/kptd/) the IMQ hook is after 'POSTROUTING'
> which means that even though I'm using '-j IMQ' in the 'mangle/forward'
> table to limit the bandwidth before counting; the counters are still 
> counting all packets; including dropped ones by IMQ.
>
> There doesn't seem to be any more 'chains' after the IMQ hook which I 
> could rely upon to 'count' the data after IMQ has done its job.
>
> I realise that when compiling the kernel, I can choose where IMQ hooks 
> in (before or after NAT); currently I have selected as 'AB'.
>
> What I'd like to know is;
>
> a) Is there something I'm missing; is there somewhere I can count the 
> packets after IMQ's work is done?
> b) If not, is there some way I can modify the IMQ hook to be 
> in-between the 'mangle/forward' and 'filter/forward' chains.
>
> Any help/comments are greatly appreciated.
>
> Thanks
> Steve.
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe lartc" in 
> the body of a message to majordomo@vger.kernel.org More majordomo info 
> at  http://vger.kernel.org/majordomo-info.html
>

Re: Hook location of IMQ

[GGounot]

Hi.

If you want to limit bandwidth to clients, I suppose the Linux box 
you're working on forwards packets to the clients. So Why do you shape 
traffic on ingress (that what I understand because you use IMQ) ? Why 
don't you use classical egress shaping ?

You must note that you cannot use iptables/mangle to mark packets going 
to IFB (I've never used IMQ) : 
http://www.mail-archive.com/lartc@mailman.ds9a.nl/msg15545.html



Le 21/10/2014 13:15, Steve (Telsat Broadband) a écrit :
> Hi All/GGounot,
>
> I've had a good review of the IFB, but it doesn't seem to have very good
> documentation on its usage/implementation (that I've found anyway).
>
> IMQ has worked very well for my purpose, but the only issue I have is where
> it is hooking.  I need a place (after PRE-ROUTING NAT) to be able to mark
> packets and then count the ones successfully delivered after they've passed
> through IMQ.
>
> The best place I could find would be to have IMQ hook in 'before' the mangle
> table in POSTROUTING.
>
> I'm not that familiar with NF hooks, but would it be possible to modify this
> in some way to have IMQ hook in before the mangle table in PostRouting?
>
>   /* imq_egress_ipv4 */
>   .hook = imq_nf_hook,
>   .owner = THIS_MODULE,
>   .pf = PF_INET,
>   .hooknum = NF_INET_POST_ROUTING,
> #if defined(CONFIG_IMQ_BEHAVIOR_AA) || defined(CONFIG_IMQ_BEHAVIOR_BA)
>   .priority = NF_IP_PRI_LAST,
> #else
>   .priority = NF_IP_PRI_NAT_SRC - 1,
> #endif
>   },
>
>
> Thanks.
> Steve.
>
>
>
> -----Original Message-----
> From: Steve (Telsat Broadband) [mailto:steve@telsatbb.vu]
> Sent: Wednesday, 17 September 2014 8:43 PM
> To: 'GGounot'; 'lartc@vger.kernel.org'
> Subject: RE: Hook location of IMQ
>
> Hi GGounot,
>
> No, to be honest, I'd never even heard of IFB.  I'm reviewing all the info
> now.
>
> Thanks very much for your reply.
>
> Thanks
> Steve
>
>
>
>
> -----Original Message-----
> From: GGounot [mailto:g.gounot@laposte.net]
> Sent: Wednesday, 17 September 2014 6:10 PM
> To: Steve (Telsat Broadband); lartc@vger.kernel.org
> Subject: Re: Hook location of IMQ
>
> Hi.
>
> Did you try IFB instead of IMQ ?
>
> "The Intermediate Functional Block device is the successor to the IMQ
> iptables module that was never integrated."
> http://www.linuxfoundation.org/collaborate/workgroups/networking/ifb
>
>
> Le 17/09/2014 01:15, Steve (Telsat Broadband) a écrit :
>> Hi All,
>>
>> I've posted a couple of questions over on linuximq.net but the
>> discussion there seems quiet, so I'll try here to see if anyone here
>> can point me in the right direction.
>>
>> Currently I use IMQ devices and TC to limit bandwidth to clients; this
>> is all working very well, except that the byte counters I'm relying on
>> for counting the clients data seems to be 'before' IMQ does its work.
>>
>> For example; I've got rules in the 'mangle/forward' table for
>> assigning the clients data to the IMQ device and rules in the
>> 'filter/forward' table which matches the client's data and I'm counting
> their traffic from here.
>> However, according to this packet flow show on linuximq.net
>> (http://www.docum.org/docum.org/kptd/) the IMQ hook is after 'POSTROUTING'
>> which means that even though I'm using '-j IMQ' in the 'mangle/forward'
>> table to limit the bandwidth before counting; the counters are still
>> counting all packets; including dropped ones by IMQ.
>>
>> There doesn't seem to be any more 'chains' after the IMQ hook which I
>> could rely upon to 'count' the data after IMQ has done its job.
>>
>> I realise that when compiling the kernel, I can choose where IMQ hooks
>> in (before or after NAT); currently I have selected as 'AB'.
>>
>> What I'd like to know is;
>>
>> a) Is there something I'm missing; is there somewhere I can count the
>> packets after IMQ's work is done?
>> b) If not, is there some way I can modify the IMQ hook to be
>> in-between the 'mangle/forward' and 'filter/forward' chains.
>>
>> Any help/comments are greatly appreciated.
>>
>> Thanks
>> Steve.
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe lartc" in
>> the body of a message to majordomo@vger.kernel.org More majordomo info
>> at  http://vger.kernel.org/majordomo-info.html
>>
>
>
>

RE: Hook location of IMQ

[Steve (Telsat Broadband)]

Hi GGounot,

Thanks very much for your reply.

Since I sent my message in, I've been attempting to modify the IMQ patch and
I've successfully moved the IMQ hook to where I want it to be called.  It's
still in testing, but so far, looks to solve my problem.  Man, I love the
power of Linux!

Thanks again for your assistance.
Steve.



-----Original Message-----
From: lartc-owner@vger.kernel.org [mailto:lartc-owner@vger.kernel.org] On
Behalf Of GGounot
Sent: Friday, 24 October 2014 7:25 AM
To: Steve (Telsat Broadband); lartc@vger.kernel.org
Subject: Re: Hook location of IMQ

Hi.

If you want to limit bandwidth to clients, I suppose the Linux box you're
working on forwards packets to the clients. So Why do you shape traffic on
ingress (that what I understand because you use IMQ) ? Why don't you use
classical egress shaping ?

You must note that you cannot use iptables/mangle to mark packets going to
IFB (I've never used IMQ) : 
http://www.mail-archive.com/lartc@mailman.ds9a.nl/msg15545.html



Le 21/10/2014 13:15, Steve (Telsat Broadband) a écrit :
> Hi All/GGounot,
>
> I've had a good review of the IFB, but it doesn't seem to have very good
> documentation on its usage/implementation (that I've found anyway).
>
> IMQ has worked very well for my purpose, but the only issue I have is
where
> it is hooking.  I need a place (after PRE-ROUTING NAT) to be able to mark
> packets and then count the ones successfully delivered after they've
passed
> through IMQ.
>
> The best place I could find would be to have IMQ hook in 'before' the
mangle
> table in POSTROUTING.
>
> I'm not that familiar with NF hooks, but would it be possible to modify
this
> in some way to have IMQ hook in before the mangle table in PostRouting?
>
>   /* imq_egress_ipv4 */
>   .hook = imq_nf_hook,
>   .owner = THIS_MODULE,
>   .pf = PF_INET,
>   .hooknum = NF_INET_POST_ROUTING,
> #if defined(CONFIG_IMQ_BEHAVIOR_AA) || defined(CONFIG_IMQ_BEHAVIOR_BA)
>   .priority = NF_IP_PRI_LAST,
> #else
>   .priority = NF_IP_PRI_NAT_SRC - 1,
> #endif
>   },
>
>
> Thanks.
> Steve.
>
>
>
> -----Original Message-----
> From: Steve (Telsat Broadband) [mailto:steve@telsatbb.vu]
> Sent: Wednesday, 17 September 2014 8:43 PM
> To: 'GGounot'; 'lartc@vger.kernel.org'
> Subject: RE: Hook location of IMQ
>
> Hi GGounot,
>
> No, to be honest, I'd never even heard of IFB.  I'm reviewing all the info
> now.
>
> Thanks very much for your reply.
>
> Thanks
> Steve
>
>
>
>
> -----Original Message-----
> From: GGounot [mailto:g.gounot@laposte.net]
> Sent: Wednesday, 17 September 2014 6:10 PM
> To: Steve (Telsat Broadband); lartc@vger.kernel.org
> Subject: Re: Hook location of IMQ
>
> Hi.
>
> Did you try IFB instead of IMQ ?
>
> "The Intermediate Functional Block device is the successor to the IMQ
> iptables module that was never integrated."
> http://www.linuxfoundation.org/collaborate/workgroups/networking/ifb
>
>
> Le 17/09/2014 01:15, Steve (Telsat Broadband) a écrit :
>> Hi All,
>>
>> I've posted a couple of questions over on linuximq.net but the
>> discussion there seems quiet, so I'll try here to see if anyone here
>> can point me in the right direction.
>>
>> Currently I use IMQ devices and TC to limit bandwidth to clients; this
>> is all working very well, except that the byte counters I'm relying on
>> for counting the clients data seems to be 'before' IMQ does its work.
>>
>> For example; I've got rules in the 'mangle/forward' table for
>> assigning the clients data to the IMQ device and rules in the
>> 'filter/forward' table which matches the client's data and I'm counting
> their traffic from here.
>> However, according to this packet flow show on linuximq.net
>> (http://www.docum.org/docum.org/kptd/) the IMQ hook is after
'POSTROUTING'
>> which means that even though I'm using '-j IMQ' in the 'mangle/forward'
>> table to limit the bandwidth before counting; the counters are still
>> counting all packets; including dropped ones by IMQ.
>>
>> There doesn't seem to be any more 'chains' after the IMQ hook which I
>> could rely upon to 'count' the data after IMQ has done its job.
>>
>> I realise that when compiling the kernel, I can choose where IMQ hooks
>> in (before or after NAT); currently I have selected as 'AB'.
>>
>> What I'd like to know is;
>>
>> a) Is there something I'm missing; is there somewhere I can count the
>> packets after IMQ's work is done?
>> b) If not, is there some way I can modify the IMQ hook to be
>> in-between the 'mangle/forward' and 'filter/forward' chains.
>>
>> Any help/comments are greatly appreciated.
>>
>> Thanks
>> Steve.
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe lartc" in
>> the body of a message to majordomo@vger.kernel.org More majordomo info
>> at  http://vger.kernel.org/majordomo-info.html
>>
>
>
>

--
To unsubscribe from this list: send the line "unsubscribe lartc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html