[LARTC] Packet marking for ingress shapping and NET

[LARTC] Packet marking for ingress shapping and NET

[Patrick Spousta]

Hi,
I have typical situation, local LAN with private addresses, translated 
via NAT to internet. I need to shape ingress traffic (from internet to 
local LAN) in several HTB queues accorting to destination (private not 
public) IP. So I need mark packets to divide them to corresponding 
queue. According to http://www.docum.org/stef.coene/qos/kptd/ I thing I 
have only one way how to do it, because MARK in PREROUTING is before (de)NAT

PREROUTING (de)NAT
               V
FORWARD    marking
               V
FORWARD   put to IMQ
               V
          HTB shapping
               V
        routing decision
               V
        output interface

It has a small problem. After PREROUTING some packets are routed to 
INPUT (packets intended for this machine for local processes)

Does exists solution how to NAT and MARK in PREROUTING, but in this order?

Patrick

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Packet marking for ingress shapping and NET

[Andreas Klauer]

Am Monday 10 May 2004 12:10 schrieb Patrick Spousta:
> So I need mark packets to divide them to corresponding queue.

That's all right so far. But the qdisc that shapes incoming traffic usually 
sits on your LAN device.

> It has a small problem. After PREROUTING some packets are routed to
> INPUT (packets intended for this machine for local processes)
>
> Does exists solution how to NAT and MARK in PREROUTING, but in this
> order?

I'm not sure if I understand what you want to do. Why do you want to mark 
INPUT packets? There is no qdisc/class to put them in. As for shaping 
incoming traffic that doesn't get forwarded to the LAN, I haven't found a 
proper solution to do that yet. So all I can do is make sure that the 
router doesn't produce any traffic (e.g. don't put a Webserver or similar 
services on it).

Andreas
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Packet marking for ingress shapping and NET

[Andy Furniss]

Patrick Spousta wrote:
> Hi,
> I have typical situation, local LAN with private addresses, translated 
> via NAT to internet. I need to shape ingress traffic (from internet to 
> local LAN) in several HTB queues accorting to destination (private not 
> public) IP. So I need mark packets to divide them to corresponding 
> queue. According to http://www.docum.org/stef.coene/qos/kptd/ I thing I 
> have only one way how to do it, because MARK in PREROUTING is before 
> (de)NAT
> 
> PREROUTING (de)NAT
>               V
> FORWARD    marking
>               V
> FORWARD   put to IMQ
>               V
>          HTB shapping
>               V
>        routing decision
>               V
>        output interface
> 
> It has a small problem. After PREROUTING some packets are routed to 
> INPUT (packets intended for this machine for local processes)
> 
> Does exists solution how to NAT and MARK in PREROUTING, but in this order?
> 

If you really need to shape for local and forwarded on ingress then you 
use IMQ + the IMQ NAT patch and use u32 to filter on dst IP (if you are 
masquerading a dynamic IP mark LAN traffic and use default for local).

If the traffic to local is not "bulk" ie just dns or ntp etc. then it 
would be less trouble to ignore it and just shape on your LAN facing 
interface marking on dst in postrouting or using u32 on dst - both 
should work, you may want to exclude traffic from server to LAN.

Andy.



_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Packet marking for ingress shapping and NAT

[Patrick Spousta]

Andreas Klauer wrote:

> Am Monday 10 May 2004 12:10 schrieb Patrick Spousta:
> 
>>So I need mark packets to divide them to corresponding queue.
> 
> 
> That's all right so far. But the qdisc that shapes incoming traffic usually 
> sits on your LAN device.

I think you are wrong. Shapping can sits on all interfaces, physical and 
logical. IMQ is logical interface.

> 
> 
>>It has a small problem. After PREROUTING some packets are routed to
>>INPUT (packets intended for this machine for local processes)
>>
>>Does exists solution how to NAT and MARK in PREROUTING, but in this
>>order?
> 
> 
> I'm not sure if I understand what you want to do. Why do you want to mark 

My linux box has 1 WAN interface (to ISP with public IP address) and 3 
LAN interfaces (with private IP addresses). Only way how to shape 
incoming traffic is use IMG device because shapping is provided on 
egress. I understood that packet 'path' looks like this

eth0 -> kernel -> IMQ -> kernel -> ethX
                       ^^^ here is 'egress' where I can do shapping.

But I need divide traffic to the corresponding queues according to real 
destination IP. Maybe I don't need marking, I can only use tc filter, 
but it must be done in place where packet has real destination IP, ie. 
behind (de)NAT.

To IMQ 'interface' I put packets via iptables. Ideal in PREROUTING 
chain, but I think I can use only 'mange' table and that is before 'nat' 
:-( So now I'm using FORWARD chain but local traffic is going outside of 
shapping path

> INPUT packets? There is no qdisc/class to put them in. As for shaping 
> incoming traffic that doesn't get forwarded to the LAN, I haven't found a 
> proper solution to do that yet. So all I can do is make sure that the 
> router doesn't produce any traffic (e.g. don't put a Webserver or similar 
> services on it).

But it isn't goor solution :-(

Patrick

> 
> Andreas
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
> 
> 

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Packet marking for ingress shapping and NET

[Patrick Spousta]

Hi

Andy Furniss wrote:

>> Does exists solution how to NAT and MARK in PREROUTING, but in this 
>> order?
>>
> 
> If you really need to shape for local and forwarded on ingress then you 
> use IMQ + the IMQ NAT patch and use u32 to filter on dst IP (if you are 
> masquerading a dynamic IP mark LAN traffic and use default for local).

It sounds good, but can you be more conrete?

> 
> If the traffic to local is not "bulk" ie just dns or ntp etc. then it 

Sometimes it is 'bulk' - FTP etc.

> would be less trouble to ignore it and just shape on your LAN facing 

Sorry, I forgot write that I have 3 LAN interfaces, so IMQ is only way 
how to do it.

Thanks
Patrick

> interface marking on dst in postrouting or using u32 on dst - both 
> should work, you may want to exclude traffic from server to LAN.
> 
> Andy.
> 


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Packet marking for ingress shapping and NET

[Andy Furniss]

Patrick Spousta wrote:
> Hi
> 
> Andy Furniss wrote:
> 
>>> Does exists solution how to NAT and MARK in PREROUTING, but in this 
>>> order?
>>>
>>
>> If you really need to shape for local and forwarded on ingress then 
>> you use IMQ + the IMQ NAT patch and use u32 to filter on dst IP (if 
>> you are masquerading a dynamic IP mark LAN traffic and use default for 
>> local).
> 
> 
> It sounds good, but can you be more conrete?

Using IMQ generally or a script? - mine is pretty lame, unfinished and 
needs netfilter patches, though I suppose it could give an indication of 
what to do - I am still learning HTB myself, but have got sidetracked at 
the moment playing with esfq.

There is a new imq website www.linuximq.net from which you should be 
able to get imq working for whatever kernel you use. I don't know if 
they include the NAT patch yet - but it's only a couple of lines and 
should apply OK. If you happen to use 2.4.24 I can give urls for the 
patches I use.

Andy.

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Packet marking for ingress shapping and NET

[Andy Furniss]

Patrick Spousta wrote:

> It looks working fine :-) I never found any details about IMQ and NAT 
> patch, it looks that packet processing in kernel has path
> 
>                 |     PREROUTING chain       |
> input interface -> contrack -> mangle -> nat -> imq

So which IMQ did you use - did you need to patch for NAT (there are 
different versions about)

> ESFQ works fine, but only for ingress shapping over imq and NAT with 
> destination hash (= download on private IPs). I'm trying to setup ESFQ 
> on egress shaping for traffic from private to public IPs with source 
> hash (upload from private) but qdisc sits after NAT, ie. packets source 
> addresses are always the same public IP of external (wan) interface :-(
> 
> I try to use imq for egress shapping (on POSTROUTING chain), may it helps

I don't think IMQ will help - but you can mark local src in postrouting 
mangle OK. If you really want to use esfq, someone posted a patch on 
here a while back which made esfq hash on fwmark.

Andy.

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Packet marking for ingress shapping and NAT

[Patrick Spousta]

Andy Furniss wrote:
> Patrick Spousta wrote:
> 
>> It looks working fine :-) I never found any details about IMQ and NAT 
>> patch, it looks that packet processing in kernel has path
>>
>>                 |     PREROUTING chain       |
>> input interface -> contrack -> mangle -> nat -> imq
> 
> 
> So which IMQ did you use - did you need to patch for NAT (there are 
> different versions about)

Now I'm using patches from http://www.digriz.org.uk/jdg-qos-script/ 
(latest version which contains patches for IMQ, IMQ+NAT, ESFQ, IPP2P, 
CONNMARK, also recompiledtc andlibrarie for iptables, nice package) on 
kernel 2.4.25. It works good.

> 
>> ESFQ works fine, but only for ingress shapping over imq and NAT with 
>> destination hash (= download on private IPs). I'm trying to setup 
>> ESFQ on egress shaping for traffic from private to public IPs with 
>> source hash (upload from private) but qdisc sits after NAT, ie. 
>> packets source addresses are always the same public IP of external 
>> (wan) interface :-(
>>
>> I try to use imq for egress shapping (on POSTROUTING chain), may it helps
> 
> 
> I don't think IMQ will help - but you can mark local src in postrouting 

Do you mean manualy configured marking for many, many IP addresses? I 
think it isn't right way :-( I like ESFQ for it's source or destination 
hash because I don't need to setup any filters/markers for those IPs, 
ESFQ creats it's own queues for each IP.

In POSTROUTING chain it normaly look like this

... -> mangle -> nat -> imq -> (output interface)

I don't understand C language so I don't understand IMQ+NAT patch, but 
I'll try to use imq for egress shapping. Maybe the patch is working 
identically on PRE i POST chains.

Patrick

> mangle OK. If you really want to use esfq, someone posted a patch on 
> here a while back which made esfq hash on fwmark.


> 
> Andy.
> 
> 

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Packet marking for ingress shapping and NAT

[Andy Furniss]

Patrick Spousta wrote:

<snip>

>> I don't think IMQ will help - but you can mark local src in postrouting 
> 
> 
> Do you mean manualy configured marking for many, many IP addresses? I 
> think it isn't right way :-( I like ESFQ for it's source or destination 
> hash because I don't need to setup any filters/markers for those IPs, 
> ESFQ creats it's own queues for each IP.

Yes I agree - not nice for your setup, though personally the thing I 
don't like about using esfq on src/dst is you loose per tcp fairness - 
it was less than a year ago that I was on 56K and anyone with high 
latency downloading from you will get their already small bandwidth 
squeezed out by the low latency downloaders.

Maybe it's less noticable/of an issue for your big setup anyway.

> 
> In POSTROUTING chain it normaly look like this
> 
> ... -> mangle -> nat -> imq -> (output interface)
> 
> I don't understand C language so I don't understand IMQ+NAT patch, but 
> I'll try to use imq for egress shapping. Maybe the patch is working 
> identically on PRE i POST chains.

I only just started getting into C myself - (used motorolla 68000 
assembly years ago on an atari ST - these "high level" languages are 
much trickier :-) ).

I can see that the patch is trivial and needs an understanding of 
netfilter hooks more than C.

I just tested with u32 to double confirm what I knew really - the patch 
only affects prerouting hooks.

Then knowing nothing about netfilter decided to have a go at changing 
the egress hook - it appears to be working as expected.

One caveat - some people have reported stability problems using 
postrouting IMQ, probably to do with dropping locally generated traffic. 
I and others don't, but then I don't leave my gateway PC up that long. 
Grepping my logs :

Sent 3744702472 bytes 5539814 pkts (dropped 354902, overlimits 11722774)

is the most I can see (most of the drops are locally generated packets 
ie. bittorrent running on the shaping PC).

If you wan't to give my blind and possibly stupid hack a go you just 
need to change near the top of drivers/net/imq.c so it looks like -

static struct nf_hook_ops imq_egress_ipv4 = {
         { NULL, NULL},
         imq_nf_hook,
         PF_INET,
         NF_IP_POST_ROUTING,
         NF_IP_PRI_NAT_SRC - 1
};


Rather than -

static struct nf_hook_ops imq_egress_ipv4 = {
         { NULL, NULL},
         imq_nf_hook,
         PF_INET,
         NF_IP_POST_ROUTING,
         NF_IP_PRI_LAST
};

You could do the same for the egress ipv6 bits below it aswell.

If you still have your source tree intact and use modules cd to top dir 
in kernel tree do

make SUBDIRS=drivers/net modules

which should build a new imq.o in drivers/net

Backup /lib/modules/[your version]/kernel/drivers/net/imq.o

and replace with new one.

Take down shaping and modprobe -r imq (check it's gone with lsmod) and 
restart shaper.

It's a bit of a pain that imq is unstable for some anyway - you won't 
know whoose fault it is if/when it crashes :-)

Andy.

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/