Re: [PATCH RFC v1 net-next 0/4] net: Qdisc backpressure infrastructure

[Peilin Ye <yepeilin.cs@gmail.com>]

Hi Eric,

On Mon, May 09, 2022 at 08:26:27PM -0700, Eric Dumazet wrote:
> On 5/6/22 12:43, Peilin Ye wrote:
> > From: Peilin Ye <peilin.ye@bytedance.com>
> > 
> > Hi all,
> > 
> > Currently sockets (especially UDP ones) can drop a lot of skbs at TC
> > egress when rate limited by shaper Qdiscs like HTB.  This experimental
> > patchset tries to improve this by introducing a backpressure mechanism, so
> > that sockets are temporarily throttled when they "send too much".
> > 
> > For now it takes care of TBF, HTB and CBQ, for UDP and TCP sockets.  Any
> > comments, suggestions would be much appreciated.  Thanks!
> 
> This very much looks like trying to solve an old problem to me.
> 
> If you were using EDT model, a simple eBPF program could get rid of the
> HTB/TBF qdisc
> 
> and you could use MQ+FQ as the packet schedulers, with the true multiqueue
> sharding.
> 
> FQ provides fairness, so a flow can not anymore consume all the qdisc limit.

This RFC tries to solve the "when UDP starts to drop (whether because of
per-flow or per-Qdisc limit), it drops a lot" issue described in [I] of
the cover letter; its main goal is not to improve fairness.

> (If your UDP sockets send packets all over the place (not connected
> sockets),
> 
> then the eBPF can also be used to rate limit them)

I was able to reproduce the same issue using EDT: default sch_fq
flow_limit (100 packets), with a 1 Gbit/sec rate limit.  Now if I run
this:

  $ iperf -u -c 1.2.3.4 -p 5678 -l 3K -i 0.5 -t 30 -b 3g

  [ ID] Interval       Transfer     Bandwidth
  [  3]  0.0- 0.5 sec   137 MBytes  2.29 Gbits/sec
  [  3]  0.5- 1.0 sec   142 MBytes  2.38 Gbits/sec
  [  3]  1.0- 1.5 sec   117 MBytes  1.96 Gbits/sec
  [  3]  1.5- 2.0 sec   105 MBytes  1.77 Gbits/sec
  [  3]  2.0- 2.5 sec   132 MBytes  2.22 Gbits/sec
  <...>                             ^^^^^^^^^^^^^^

On average it tries to send 2.31 Gbits per second, dropping 56.71% of
the traffic:

  $ tc -s qdisc show dev eth0
  <...>
  qdisc fq 5: parent 1:4 limit 10000p flow_limit 100p buckets 1024 orphan_mask 1023 quantum 18030 initial_quantum 90150 low_rate_threshold 550Kbit refill_delay 40.0ms
   Sent 16356556 bytes 14159 pkt (dropped 2814461, overlimits 0 requeues 0)
                                  ^^^^^^^^^^^^^^^

This RFC does not cover EDT though, since it does not use Qdisc watchdog
or friends.

Thanks,
Peilin Ye