tc question about ingress bandwidth splitting

tc question about ingress bandwidth splitting

[Philip Prindeville]

Hi all,

I asked around on IRC but no one seems to know the answer, so I thought I’d go to the source… Seemed like something Stephen or Eric might be able to answer.

I have a SoHo router with two physical subnets, which we’ll call “production” (eth0) and “guest” (eth1), and the egress interface “wan” (eth5).

The uplink is G.PON 50/10 mbps.  I’d like to cap the usage on “guest” to 10/2 mbps.  Any unused bandwidth from “guest” goes to “production”.

I thought about marking the traffic coming in off “wan" (the public interface).  Then using HTB to have a 50 mbps cap at the root, and allocating 10mb/s to the child “guest”.  The other sibling would be “production”, and he gets the remaining traffic.

Upstream would be the reverse, marking ingress traffic from “guest” with a separate tag.  Allocating upstream root on “wan” with 10 mbps, and the child “guest” getting 2 mbps.  The remainder goes to the sibling “production”.

Should be straightforward enough, right? (Well, forwarding is more straightforward than traffic terminating on the router itself, I guess… bonus points for getting that right, too.)

I’m hoping that the limiting will work adequately so that the end-to-end path has adequate congestion avoidance happening, and that upstream doesn’t overrun the receiver and cause a lot of packets to be dropped on the last hop (work case of wasted bandwidth).  Not sure if I need special accommodations for bursting or if that would just delay the “settling” of congestion avoidance into steady-state.

Also not sure if ECN is worth marking at this point.  Congestion control is supposed to work better than congestion avoidance, right?

Anyone know what the steps would look like to accomplish the above?

A bunch of people responded, “yeah, I’ve been wanting to do that too…” when I brought up my question, so if I get a good solution I’ll submit a FAQ entry to LARC.

Thanks,

-Philip

Re: tc question about ingress bandwidth splitting

[Gáspár Lajos]

Hi Philip,


Just a tip: AFAIK, you can only limit your sending bandwith... 
Everything what you already received is already on your device... :)


Cheers,

Lajos


2020. 03. 22. 19:20 keltezéssel, Philip Prindeville írta:
> Hi all,
>
> I asked around on IRC but no one seems to know the answer, so I thought I’d go to the source… Seemed like something Stephen or Eric might be able to answer.
>
> I have a SoHo router with two physical subnets, which we’ll call “production” (eth0) and “guest” (eth1), and the egress interface “wan” (eth5).
>
> The uplink is G.PON 50/10 mbps.  I’d like to cap the usage on “guest” to 10/2 mbps.  Any unused bandwidth from “guest” goes to “production”.
>
> I thought about marking the traffic coming in off “wan" (the public interface).  Then using HTB to have a 50 mbps cap at the root, and allocating 10mb/s to the child “guest”.  The other sibling would be “production”, and he gets the remaining traffic.
>
> Upstream would be the reverse, marking ingress traffic from “guest” with a separate tag.  Allocating upstream root on “wan” with 10 mbps, and the child “guest” getting 2 mbps.  The remainder goes to the sibling “production”.
>
> Should be straightforward enough, right? (Well, forwarding is more straightforward than traffic terminating on the router itself, I guess… bonus points for getting that right, too.)
>
> I’m hoping that the limiting will work adequately so that the end-to-end path has adequate congestion avoidance happening, and that upstream doesn’t overrun the receiver and cause a lot of packets to be dropped on the last hop (work case of wasted bandwidth).  Not sure if I need special accommodations for bursting or if that would just delay the “settling” of congestion avoidance into steady-state.
>
> Also not sure if ECN is worth marking at this point.  Congestion control is supposed to work better than congestion avoidance, right?
>
> Anyone know what the steps would look like to accomplish the above?
>
> A bunch of people responded, “yeah, I’ve been wanting to do that too…” when I brought up my question, so if I get a good solution I’ll submit a FAQ entry to LARC.
>
> Thanks,
>
> -Philip
>

Re: tc question about ingress bandwidth splitting

[Marc SCHAEFER]

On Mon, Mar 23, 2020 at 07:47:11AM +0100, Gáspár Lajos wrote:
> Just a tip: AFAIK, you can only limit your sending bandwith... Everything
> what you already received is already on your device... :)

Right, however in case you do not have control on the sending
device, you can slow down or loose TCP ACKs to slow down
the opposite direction TCP traffic.

Re: tc question about ingress bandwidth splitting

[Philip Prindeville]

> On Mar 23, 2020, at 3:36 AM, Marc SCHAEFER <schaefer@alphanet.ch> wrote:
> 
> On Mon, Mar 23, 2020 at 07:47:11AM +0100, Gáspár Lajos wrote:
>> Just a tip: AFAIK, you can only limit your sending bandwith... Everything
>> what you already received is already on your device... :)
> 
> Right, however in case you do not have control on the sending
> device, you can slow down or loose TCP ACKs to slow down
> the opposite direction TCP traffic.


Exactly.  That’s my desire.

The sender calibrates based on 4 observed end-to-end path properties:

1. min path MTU, i.e. the smallest of all per-hop path MTU’s (likely doesn’t come into play here);
2. end-to-end delay, i.e. the summation of all per-hop delays, which we can artificially increase (shape) on the final internal hop to affect sender pacing;
3. end-to-end bandwidth, again the minimum bandwidth for the slowest hop along the path, which we can also artificially increase on the final internal hop to affect sender pacing;
4. end-to-end loss, the product of the reliability of all per-hop lossiness (hopefully we won’t be increasing lossiness, with the last hop having a reliability of 1.0, assuming that delay provides enough back pressure that we don’t need to actually drop any packets);

So the intention is to use (2) and (3) to apply back-pressure to the sender to influence his congestion avoidance.

Hopefully that makes things a little more clear!

So, back to my main question… what do the tc/htb commands look like to configure a tree that’s spread out across multiple (well, two in this case) interfaces?  Is that even possible?

I guess I could do all of the shaping on the “wan” interface, including on ingress… and just use the destination subnet (post NATting, obviously) on ingress, and based on the source subnet or “indev” on egress.

Thanks,

-Philip