Are there any alternatives to IFB for downlink shaping?

Are there any alternatives to IFB for downlink shaping?

[Nick Pelling]

Hi everyone,

I'm trying to configure (OK, 'hack') a Linux-based Broadcom wireless gateway to prioritize the traffic between two groups of interfaces, e.g. where home traffic gets rate 80% ceil 95% prio 1, and where guest traffic (i.e. wl1) gets rate 15% ceil 60% prio 2.

I now have tc doing this beautifully on the uplink traffic (I use 'action skbedit mark 1', why is this trick mentioned hardly anywhere?): but, like almost every other first time poster here :-) , I'm having no luck at all getting this working for downlink traffic.

Specifically, I've put in a lot of work trying to get IFB working, but it seems to be stitched too early in the packet processing chain to be any use for shaping a WAN interface's downlink traffic. Basically, my downlink bandwidth stats stay resolutely at zero bytes / zero  packets, whatever I try. :-(

Are there any higher-level (i.e. device-level) configuration alternatives to using IFB? It seems obvious to me that if I could insert any (fake, forwarding-only) device at all between the WAN port and the rest of the system, then I would be able to use normal  tc egress shaping on the traffic going through it.

It strikes me that this would be something a tap device could do, for example: but I've never seen anyone discuss this anywhere on the web.

Anyway, I'm currently using eth3 for the WAN, so my (working) uplink traffic shaping looks like this:-

eth0-2 / wl0 ---> br0 --->  \
                                                     +---> [tc egress] eth3 --> [Internet]
wl1                                  --->  /

For the downlink, however, I'm wondering if there might be a fake device I could configure in that serves only to forward packets. tap0?

eth0-2 / wl0 <--- br0 <---  \
                                                     + <--- fakedevice [tc egress] <--- eth3 <--- [Internet]
wl1                                   <---  /

I was hoping (of course) that IFB would be able to do this for me, but it doesn't seem to want to play nicely. Am I missing something? All suggestions and comments very much appreciated! :-)

Thanks very much, Nick

Re: Are there any alternatives to IFB for downlink shaping?

[Marco Gaiarin]

Mandi! Nick Pelling
  In chel di` si favelave...

> I now have tc doing this beautifully on the uplink traffic (I use 'action skbedit mark 1', why is this trick mentioned hardly anywhere?): but, like almost every other first time poster here :-) , I'm having no luck at all getting this working for downlink traffic.

?! Looking on google form 'skbedit' lead me to multiqueue management:

	https://www.kernel.org/doc/Documentation/networking/multiqueue.txt

and seems not related... can you explain better?


> Specifically, I've put in a lot of work trying to get IFB working, but it seems to be stitched too early in the packet processing chain to be any use for shaping a WAN interface's downlink traffic. Basically, my downlink bandwidth stats stay resolutely at zero bytes / zero  packets, whatever I try. :-(

I've asked something similar some weeks ago, here.

I've also done some tests, and seems to me that, missing iptables
hooks, you have also no NAT, so if you have to match dest IP you cannot
in IFB, because dest ip is everytime the public ip. ;(

-- 
dott. Marco Gaiarin				        GNUPG Key ID: 240A3D66
  Associazione ``La Nostra Famiglia''                    http://www.sv.lnf.it/
  Polo FVG   -   Via della Bontà, 7 - 33078   -   San Vito al Tagliamento (PN)
  marco.gaiarin(at)lanostrafamiglia.it   t +39-0434-842711   f +39-0434-842797

		Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA!
    http://www.lanostrafamiglia.it/25/index.php/component/k2/item/123
	(cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA)

Re: Are there any alternatives to IFB for downlink shaping?

[Mike Schmidt]

We use ifb in both directions, and it works very well.  Bur Marco is
right, matching the dest ip within ifb will not work if you are
natted. However, I'm sure there are ways around that. Here's an
example of what we do (the client devices come in on br0):

the ingress device is ifb0, the egress is ifb1

echo load br0 egress classes
/sbin/tc class add dev br0 parent 1: classid 1:1 hfsc sc rate 1000mbit
/sbin/tc class add dev br0 parent 1:1 classid 1:10 hfsc sc rate 1000mbit
/sbin/tc filter add dev br0 parent 1: protocol ip priority 10 u32
match u32 0 0 flowid 1:10 action mirred egress redirect dev ifb1

echo load br0 ingress qdisc
/sbin/tc qdisc del dev br0 handle ffff: ingress  2>/dev/null
/sbin/tc qdisc add dev br0 handle ffff: ingress
/sbin/tc filter add dev br0 parent ffff: protocol ip priority 10 u32
match u32 0 0 flowid 1: action mirred egress redirect dev ifb0

# then, just provide the regular qdiscs, classes and filters to ifb0 and ifb1.

This works well for us; hfsc provides a fairly solid bandwidth control
in our case. The only downside is that we need to set up the same for
all client networks (we have a number of vlans on that side besides
br0). Note that the ifb devices are attached not on the wan
interfaces, but on the client side interfaces.



On Mon, Mar 16, 2015 at 12:26 PM, Marco Gaiarin <gaio@sv.lnf.it> wrote:
> Mandi! Nick Pelling
>   In chel di` si favelave...
>
>> I now have tc doing this beautifully on the uplink traffic (I use 'action skbedit mark 1', why is this trick mentioned hardly anywhere?): but, like almost every other first time poster here :-) , I'm having no luck at all getting this working for downlink traffic.
>
> ?! Looking on google form 'skbedit' lead me to multiqueue management:
>
>         https://www.kernel.org/doc/Documentation/networking/multiqueue.txt
>
> and seems not related... can you explain better?
>
>
>> Specifically, I've put in a lot of work trying to get IFB working, but it seems to be stitched too early in the packet processing chain to be any use for shaping a WAN interface's downlink traffic. Basically, my downlink bandwidth stats stay resolutely at zero bytes / zero  packets, whatever I try. :-(
>
> I've asked something similar some weeks ago, here.
>
> I've also done some tests, and seems to me that, missing iptables
> hooks, you have also no NAT, so if you have to match dest IP you cannot
> in IFB, because dest ip is everytime the public ip. ;(
>
> --
> dott. Marco Gaiarin                                     GNUPG Key ID: 240A3D66
>   Associazione ``La Nostra Famiglia''                    http://www.sv.lnf.it/
>   Polo FVG   -   Via della Bontà, 7 - 33078   -   San Vito al Tagliamento (PN)
>   marco.gaiarin(at)lanostrafamiglia.it   t +39-0434-842711   f +39-0434-842797
>
>                 Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA!
>     http://www.lanostrafamiglia.it/25/index.php/component/k2/item/123
>         (cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA)
> --
> To unsubscribe from this list: send the line "unsubscribe lartc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Mike SCHMIDT
CTO
Intello Technologies Inc.
mike.schmidt@intello.com
Canada: 1-888-404-6261 x320
USA: 1-888-404-6268 x320
Mobile: 514-409-6898
www.intello.com

Re: Are there any alternatives to IFB for downlink shaping?

[Andrew Beverley]

On Mon, 2015-03-16 at 16:00 +0000, Nick Pelling wrote:
> Specifically, I've put in a lot of work trying to get IFB working, but
> it seems to be stitched too early in the packet processing chain to be
> any use for shaping a WAN interface's downlink traffic.

Correct, the packets will hit IFB before you've been able to do anything
useful to them such as marking them. If you want to hook into a more
useful location, then you can use IMQ, but you'll need to patch your
kernel:

https://github.com/imq/linuximq/wiki/WhatIs

> Are there any higher-level (i.e. device-level) configuration
> alternatives to using IFB?

The best alternative is to do egress shaping on your LAN device instead.
You can use appropriate rules to ensure you're only shaping packets that
have originated from the WAN connection. This will have the same effect,
although it relies on you forwarding all your traffic, as you can't
shape anything that's destined for the localhost.

Disclaimer: it's been a while since I played with any of the above, so
things may have changed.

Re: Are there any alternatives to IFB for downlink shaping?

[Andy Furniss]

Nick Pelling wrote:

> I was hoping (of course) that IFB would be able to do this for me,
> but it doesn't seem to want to play nicely. Am I missing something?
> All suggestions and comments very much appreciated! :-)

I think IFB should do here, Andrew already mentioned IMQ, but for normal
setups the only real need for that would be if you need to shape traffic
destined for the router as well as forwarded and then only if you have
to nat onto a single public address. Your setup seems to be just
forwarding so that doesn't apply.

You should be able to use ifb to catch traffic leaving on the lan facing
egress devices. Having previously marked using iptables
depending on dst if or mark with tc.

To use filters to redirect from both br0 and wl1 to a single ifb you
will need to add eg, a prio qdisc to br0 and wl1 (just because you can't
add filters to a device without a "real" qdisc on it, the default
pfifo_fast will not do).

One other thing to remember is that shaping traffic coming in from wan
is not the same as shaping that going out - you are at the wrong end of
the bottleneck and have to back off/sacrifice some bandwidth or you will
never build up a queue to do anything with. How much to back off
somewhat depends on the speed of the link and how much you care about
latency and how hard you hit the line eg shaping a few tcps is easier
than a few hundred from torrents.

Re: Are there any alternatives to IFB for downlink shaping?

[Nick Pelling]

Hi Marco,

"action skbedit" is a simple tc action that edits skb packets in fairly straightforward ways, e.g. "action skbedit mark 1" edits the filtered packet to have mark = 1.

Generally, I prefer "tc filter" commands to "iptables -A" commands, mainly because the former are bound to an individual interface rather than to every netfilter chain call ever. :-)

________________________________________
From: lartc-owner@vger.kernel.org <lartc-owner@vger.kernel.org> on behalf of Marco Gaiarin <gaio@sv.lnf.it>
Sent: 16 March 2015 16:26
To: lartc@vger.kernel.org
Subject: Re: Are there any alternatives to IFB for downlink shaping?

> I now have tc doing this beautifully on the uplink traffic (I use 'action skbedit mark 1', why is this trick mentioned hardly anywhere?): but, like almost every other first time poster here :-) , I'm having no luck at all getting this working for downlink traffic.

?! Looking on google form 'skbedit' lead me to multiqueue management:

        https://www.kernel.org/doc/Documentation/networking/multiqueue.txt

and seems not related... can you explain better?

Re: Are there any alternatives to IFB for downlink shaping?

[Nick Pelling]

Thanks very much for your quick replies, Andrew and Andy. :-)

In short: I'm trying to prioritize bandwidth between two groups of interfaces, so that users in the home group get priority over users in the guest group, all the while sharing the uplink bandwidth and downlink bandwidth nicely (i.e. nobody can monopolize the uplink or downlink bandwidths etc).

And yes: in an ideal world, downlink traffic would be shaped at the other end of the DSL line (because by the time packets have arrived, the downlink bandwidth has already been hogged), hence downlink shaping is indeed "on the wrong end of the bottleneck".

But it seems that the only 'control lever' a home/guest Linux wireless gateway could 'pull' to try to discourage such downlink bandwidth hogs would be delaying or perhaps even dropping their packets, even if those packets arrived OK. And though that's not a great answer, that's just about the only answer I can see.

All the same, thanks for your specific suggestions: I shall go and try them out and hopefully come back when they're all working as well as the uplink shaping. ;-)

My $0.02's worth: I suspect that the ongoing explosion in WiFi devices means that QoS and local traffic management are now about to be reluctantly pushed centre stage, because the humble wireless gateway now has all kinds of real-time traffic competing and colliding. The R&D I'm doing at the moment is just a small part of this very much wider landscape: but I do wonder whether the current set of tc qdiscs and filters will prove to be up to the job.