Ingress filter is overly aggressive

Ingress filter is overly aggressive

[Neil Aggarwal]

Hello:

I have these rules set up for an interface:

IP=a.b.c.d
DEV=v1208
/sbin/tc qdisc del dev $DEV root
/sbin/tc qdisc add dev $DEV root handle 1: htb default 30
/sbin/tc class add dev $DEV parent 1: classid 1:1 htb rate 5mbit
/sbin/tc class add dev $DEV parent 1: classid 1:2 htb rate 5mbit
/sbin/tc filter add dev $DEV protocol ip parent 1:0 prio 1 u32 match ip dst
$IP flowid 1:1
/sbin/tc filter add dev $DEV protocol ip parent 1:0 prio 1 u32 match ip src
$IP flowid 1:2
/sbin/tc qdisc del dev $DEV ingress
/sbin/tc qdisc add dev $DEV ingress handle ffff:
/sbin/tc filter add dev $DEV parent ffff: protocol ip prio 50 u32 match ip
src 0.0.0.0/0 police rate 5mbit burst 500k drop flowid :1

According to these rules, the ingress police rate should be 5 mbit/sec, but
I am barely
getting 50 kbits/sec through the interface.

If I remove the filter, the interface operates at full line rate.

Any ideas why the ingress filter is being so aggressive?

Thanks,
  Neil

--
Neil Aggarwal, (972)834-1565, http://UnmeteredVPS.net/centos
Virtual private server with CentOS 6 preinstalled
Unmetered bandwidth = no overage charges

Re: Ingress filter is overly aggressive

[Aleš Rataj]

2013/2/19 Neil Aggarwal <neil@jammconsulting.com>:
> Hello:
>
> I have these rules set up for an interface:
>
> IP=a.b.c.d
> DEV=v1208
> /sbin/tc qdisc del dev $DEV root
> /sbin/tc qdisc add dev $DEV root handle 1: htb default 30
> /sbin/tc class add dev $DEV parent 1: classid 1:1 htb rate 5mbit
> /sbin/tc class add dev $DEV parent 1: classid 1:2 htb rate 5mbit
> /sbin/tc filter add dev $DEV protocol ip parent 1:0 prio 1 u32 match ip
> dst
> $IP flowid 1:1
> /sbin/tc filter add dev $DEV protocol ip parent 1:0 prio 1 u32 match ip
> src
> $IP flowid 1:2
> /sbin/tc qdisc del dev $DEV ingress
> /sbin/tc qdisc add dev $DEV ingress handle ffff:
> /sbin/tc filter add dev $DEV parent ffff: protocol ip prio 50 u32 match ip
> src 0.0.0.0/0 police rate 5mbit burst 500k drop flowid :1
>
> According to these rules, the ingress police rate should be 5 mbit/sec,
> but
> I am barely
> getting 50 kbits/sec through the interface.
>
> If I remove the filter, the interface operates at full line rate.
>
> Any ideas why the ingress filter is being so aggressive?
>
> Thanks,
>   Neil

You need to set the parameter CEIL in definition of HTB,

i.e. /sbin/tc class add dev $DEV parent 1: classid 1:2 htb rate
0,5mbit ceil 5mbit

The ceil argument specifies the maximum bandwidth that a class can use.

RE: Ingress filter is overly aggressive

[Neil Aggarwal]

Ales:

>> I am barely
>> getting 50 kbits/sec through the interface.

> i.e. /sbin/tc class add dev $DEV parent 1: classid 1:2 htb rate
> 0,5mbit ceil 5mbit
> The ceil argument specifies the maximum bandwidth that a class can use.

The egress filter is not a problem.  It is filtering at exactly 5 mbit/sec
as I 
want.  The problem is on the ingress filter.

Thanks,
  Neil

--
Neil Aggarwal, (972)834-1565, http://UnmeteredVPS.net/centos
Virtual private server with CentOS 6 preinstalled
Unmetered bandwidth = no overage charges

Re: Ingress filter is overly aggressive

[Andy Furniss]

Neil Aggarwal wrote:
> Hello:
>
> I have these rules set up for an interface:
>
> IP=a.b.c.d
> DEV=v1208
> /sbin/tc qdisc del dev $DEV root
> /sbin/tc qdisc add dev $DEV root handle 1: htb default 30
> /sbin/tc class add dev $DEV parent 1: classid 1:1 htb rate 5mbit
> /sbin/tc class add dev $DEV parent 1: classid 1:2 htb rate 5mbit
> /sbin/tc filter add dev $DEV protocol ip parent 1:0 prio 1 u32 match ip dst
> $IP flowid 1:1
> /sbin/tc filter add dev $DEV protocol ip parent 1:0 prio 1 u32 match ip src
> $IP flowid 1:2
> /sbin/tc qdisc del dev $DEV ingress
> /sbin/tc qdisc add dev $DEV ingress handle ffff:
> /sbin/tc filter add dev $DEV parent ffff: protocol ip prio 50 u32 match ip
> src 0.0.0.0/0 police rate 5mbit burst 500k drop flowid :1
>
> According to these rules, the ingress police rate should be 5 mbit/sec, but
> I am barely
> getting 50 kbits/sec through the interface.
>
> If I remove the filter, the interface operates at full line rate.
>
> Any ideas why the ingress filter is being so aggressive?

Hmm, I just tested and got the same.

It seems the problem is that you don't specify mtu adding mtu 3100 fixed 
for me.

I think there is an issue with policer and kernel receive offload - 
which on my box at least makes it look like double size packets are 
arriving (you can turn it off with ethtool).

Last time I looked with ifb it seemed that anything on that actually saw 
the packets as normal size (1514 on eth).

Testing above policer with mtu 1600 doesn't work though (well it works 
slowly as I guess tcp is eventually beaten into sending one packet at a 
time), so it seems policer is seeing double size phony packets as mtu 
3100 does work.

Policers can be over aggresive even when working eg. though 500k burst 
seemed to work for one file testing on 100mbit eth I don't know if It 
would be too big for many connections - but then setting too small is 
also likely to hurt - needs lots of testing with real workloads I guess.

RE: Ingress filter is overly aggressive

[Neil Aggarwal]

Andy:

> It seems the problem is that you don't specify mtu adding mtu 3100 fixed 
> for me.

You are right.  Setting the MTU to a high number fixed the problem.

Thank you,
  Neil

--
Neil Aggarwal, (972)834-1565, http://UnmeteredVPS.net/centos
Virtual private server with CentOS 6 preinstalled
Unmetered bandwidth = no overage charges

RE: Ingress filter is overly aggressive

[Neil Aggarwal]

Andy:

> It seems the problem is that you don't specify mtu adding 
> mtu 3100 fixed for me.

The MTU setting is extremely strange.

When I left it at the default value, the ingress was limited to a sustained
rate of about 100 kbit/sec.

When I set the MTU to 3100 as you suggested, the ingress had a sustained
rate about 6 mbit/sec which is better but still not the target I wanted.

When I set the MTU to 5k, the ingress has a sustained rate about 2.6
mbit/sec
which I find counter-intuitive since it does not seem to be a linear
relationship.

Does anyone have any insights into this?

Thanks,
  Neil

--
Neil Aggarwal, (972)834-1565, http://UnmeteredVPS.net/centos
Virtual private server with CentOS 6 preinstalled
Unmetered bandwidth = no overage charges

Re: Ingress filter is overly aggressive

[Andy Furniss]

Neil Aggarwal wrote:
> Andy:
>
>> It seems the problem is that you don't specify mtu adding
>> mtu 3100 fixed for me.
>
> The MTU setting is extremely strange.
>
> When I left it at the default value, the ingress was limited to a sustained
> rate of about 100 kbit/sec.
>
> When I set the MTU to 3100 as you suggested, the ingress had a sustained
> rate about 6 mbit/sec which is better but still not the target I wanted.
>
> When I set the MTU to 5k, the ingress has a sustained rate about 2.6
> mbit/sec
> which I find counter-intuitive since it does not seem to be a linear
> relationship.
>
> Does anyone have any insights into this?

http://ace-host.stuart.id.au/russell/files/tc/doc/

There is some info in the police.txt there, it's old and I don't know 
how valid it is.

I've never used/tested policers with anything but the simplest of params.

I've also never tested any tc on virtual setups.