Does tc-prio really work as advertised?

Does tc-prio really work as advertised?

[Joerg Pommnitz]

Hello all,
I might make a fool out of me, but I think the prio qdisc doesn't work as advertised in any document I could lay my hands on.

My problem was that the link quality reported by the olsr.org olsrd degraded depending on the amount of payload traffic transferred through an adhoc/mesh interface. The LQ is calculated from the packet loss of LQ Hello packets sent through this interface. To make sure normal traffic does not interfere with this value, olsrd sets the TOS field to 0x10 (Minimize-Delay) by default. This should give olsr traffic the highest priority on the link.

Investigating this issue I replaced the default Pfifo_fast with a prio qdisc and attached a pfifo on each of the bands:

    INTERFACE=wifi0
    tc qdisc add dev $INTERFACE root handle 1: prio
    tc qdisc add dev $INTERFACE parent 1:1 handle 10: pfifo
    tc qdisc add dev $INTERFACE parent 1:2 handle 20: pfifo
    tc qdisc add dev $INTERFACE parent 1:3 handle 30: pfifo

The I used ping -Q TOSVALUE to send packets with different TOS values through the interface. tcpdump confirmed the correct TOS values in the outgoing packets.

With "tc -s qdisc ls dev wifi0" I could observe the effects of the different TOS values. The result: no effect at all! Every single packet used the band indicated by the first value in the priomap (e.g. band 1 by default, in my case the pfifo with handle 20:). I can't square this observation with the available documentation.

Looking at the source code, it seems that sched_prio uses the skb->priority value to select the outgoing band. According to some documentation I found, an application can set this value.

Now I'm at a loss. I can work around this problem with filters, but I don't think that this is the correct solution. Any suggestions?

 






      Heute schon einen Blick in die Zukunft von E-Mails wagen? www.yahoo.de/mail

-- 
Olsr-users mailing list
Olsr-users@lists.olsr.org
http://lists.olsr.org/mailman/listinfo/olsr-users

Re: Does tc-prio really work as advertised?

[Jarek Poplawski]

Joerg Pommnitz wrote, On 11/23/2007 03:47 PM:

> Hello all,
> I might make a fool out of me, but I think the prio qdisc doesn't work as advertised in any document I could lay my hands on.


Marketing?

> 
> My problem was that the link quality reported by the olsr.org olsrd degraded depending on the amount of payload traffic transferred through an adhoc/mesh interface. The LQ is calculated from the packet loss of LQ Hello packets sent through this interface. To make sure normal traffic does not interfere with this value, olsrd sets the TOS field to 0x10 (Minimize-Delay) by default. This should give olsr traffic the highest priority on the link.
> 
> Investigating this issue I replaced the default Pfifo_fast with a prio qdisc and attached a pfifo on each of the bands:
> 
>     INTERFACE=wifi0
>     tc qdisc add dev $INTERFACE root handle 1: prio
>     tc qdisc add dev $INTERFACE parent 1:1 handle 10: pfifo
>     tc qdisc add dev $INTERFACE parent 1:2 handle 20: pfifo
>     tc qdisc add dev $INTERFACE parent 1:3 handle 30: pfifo
> 
> The I used ping -Q TOSVALUE to send packets with different TOS values through the interface. tcpdump confirmed the correct TOS values in the outgoing packets.
> 
> With "tc -s qdisc ls dev wifi0" I could observe the effects of the different TOS values. The result: no effect at all! Every single packet used the band indicated by the first value in the priomap (e.g. band 1 by default, in my case the pfifo with handle 20:). I can't square this observation with the available documentation.
> 
> Looking at the source code, it seems that sched_prio uses the skb->priority value to select the outgoing band. According to some documentation I found, an application can set this value.
> 
> Now I'm at a loss. I can work around this problem with filters, but I don't think that this is the correct solution. Any suggestions?
> 

Are you doing this on the same box? I was tracing this long time ago too, and, if
I didn't miss something, it was about the place! So, as I recall (after finding
some old message) this TOS is considered only for packets going through the FORWARD
chain. (But, I haven't checked this at all now, so "no complaints"...)

Regards,
Jarek P.

Re: Does tc-prio really work as advertised?

[Jarek Poplawski]

On 26-11-2007 23:25, Jarek Poplawski wrote:
...
> Are you doing this on the same box? I was tracing this long time ago too, and, if
> I didn't miss something, it was about the place! So, as I recall (after finding
> some old message) this TOS is considered only for packets going through the FORWARD
> chain. (But, I haven't checked this at all now, so "no complaints"...)

...Too exactly! Iptables aren't needed for this, so "going through
the forward." should be enough...

Jarek P.

RE: Does tc-prio really work as advertised?

[Joerg Pommnitz]

Jarek,
iptables chains (this is what I think you are referring to) are not the issue. This
is about the qdisc that sits immediately over the device driver and decides the
order waiting packets are sent over the line/air/carrier pigeon/... .
My suspicion is that skb->priority used to be set to a value that derived from the
TOS bits. Then something changed and nobody noticed.
-- 
 
Regards
 
       Joerg
 


----- Ursprüngliche Mail ----
Von: Jarek Poplawski <jarkao2@o2.pl>
An: Jarek Poplawski <jarkao2@o2.pl>
CC: Joerg Pommnitz <pommnitz@yahoo.com>; netdev@vger.kernel.org; OLSR discussion and development <olsr-users@olsr.org>
Gesendet: Dienstag, den 27. November 2007, 07:46:04 Uhr
Betreff: Re: Does tc-prio really work as advertised?

On 26-11-2007 23:25, Jarek Poplawski wrote:
...
> Are you doing this on the same box? I was tracing this long time ago  too, and, if
> I didn't miss something, it was about the place! So, as I recall  (after finding
> some old message) this TOS is considered only for packets going  through the FORWARD
> chain. (But, I haven't checked this at all now, so "no  complaints"...)

...Too exactly! Iptables aren't needed for this, so "going through
the forward." should be enough...

Jarek P.






      Machen Sie Yahoo! zu Ihrer Startseite. Los geht's: 
http://de.yahoo.com/set

Re: Does tc-prio really work as advertised?

[Jarek Poplawski]

On Tue, Nov 27, 2007 at 01:28:43AM -0800, Joerg Pommnitz wrote:
> Jarek,
> iptables chains (this is what I think you are referring to) are not the issue.

Yes, but this could (wrongly) look like this according to my 1-st message.

> This
> is about the qdisc that sits immediately over the device driver and decides the
> order waiting packets are sent over the line/air/carrier pigeon/... .
> My suspicion is that skb->priority used to be set to a value that derived from the
> TOS bits. Then something changed and nobody noticed.

I'm not sure of your problem: did you try this on a box which
gets packets with TOS set earlier, does forwarding, and uses this
prio on egress? If so, and this doesn't work, then you are right
something could be wrong.

Regards,
Jarek P.

Re: Does tc-prio really work as advertised?

[Jarek Poplawski]

On Tue, Nov 27, 2007 at 02:54:10AM -0800, Joerg Pommnitz wrote:
> Jarek,
> this is all about outgoing packets, e.g. egress to use your word.
> It doesn't matter whether the packets are originated locally or
> whether the packets are forwarded from another host (I tried
> both).
> 
> To restate the problem: according to my observations the prio qdisc
> (and probably pfifo_fast, but I couldn't observe this) does not prioritize
> at all and always uses the band indicated by the first entry in the
> priomap.
> 
> By default the priomap looks like this:
> qdisc prio 1: dev eth1 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
> 
> there are 3 bands (1:1, 1:2 and 1:3). In theory the traffic should go through
> the different bands according to the TOS value of the packets. My observation
> is, that the traffic always uses the band pointed to by the first entry in the
> priomap. This value is 1 by default, so all traffic goes through band 1:2.
> 
> Now it's entirely possible that I did something stupid, but nobody came forward
> to show me the error of my ways (neither here nor on the lartc list).
> 

I don't think there could be anything stupid if something is maybe not
enough documented. But, this really should work - just like you've
found: TOS should be recalculated to skb->priority, and this should
affect prio. You should only consider that this recalculation isn't
done for all packets, but only forwarded ones (if I can remember, didn't
miss something, and nothing changed later...). So, are you still sure
you've tested such a case? (Btw., there are some other tools which can
change this priority field, so I hope you don't use them too.)

Jarek P.

Re: Does tc-prio really work as advertised?

[Patrick McHardy]

Jarek Poplawski wrote:
> On Tue, Nov 27, 2007 at 02:54:10AM -0800, Joerg Pommnitz wrote:
>> Jarek,
>> this is all about outgoing packets, e.g. egress to use your word.
>> It doesn't matter whether the packets are originated locally or
>> whether the packets are forwarded from another host (I tried
>> both).
>>
>> To restate the problem: according to my observations the prio qdisc
>> (and probably pfifo_fast, but I couldn't observe this) does not prioritize
>> at all and always uses the band indicated by the first entry in the
>> priomap.
>>
>> By default the priomap looks like this:
>> qdisc prio 1: dev eth1 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
>>
>> there are 3 bands (1:1, 1:2 and 1:3). In theory the traffic should go through
>> the different bands according to the TOS value of the packets. My observation
>> is, that the traffic always uses the band pointed to by the first entry in the
>> priomap. This value is 1 by default, so all traffic goes through band 1:2.
>>
>> Now it's entirely possible that I did something stupid, but nobody came forward
>> to show me the error of my ways (neither here nor on the lartc list).
>>
> 
> I don't think there could be anything stupid if something is maybe not
> enough documented. But, this really should work - just like you've
> found: TOS should be recalculated to skb->priority, and this should
> affect prio. You should only consider that this recalculation isn't
> done for all packets, but only forwarded ones (if I can remember, didn't
> miss something, and nothing changed later...). So, are you still sure
> you've tested such a case? (Btw., there are some other tools which can
> change this priority field, so I hope you don't use them too.)


It works fine here, I'm guessing that Jörg is using an old kernel
version that had a bug in prio classification without filters.

Re: Does tc-prio really work as advertised?

[Joerg Pommnitz]

Great :-(. I went over the ChangeLogs for later kernel versions, but
couldn't find anything.

I'll try to come up with a working .config for the most current kernel
that works on my hardware and test again.

Do you have a pointer to the fix so that I could try to patch a 2.6.20
kernel?
 
--  
Regards
 
       Joerg

----- Ursprüngliche Mail ----
Von: Patrick McHardy <kaber@trash.net>
An: Joerg Pommnitz <pommnitz@yahoo.com>
CC: Jarek Poplawski <jarkao2@o2.pl>; netdev@vger.kernel.org
Gesendet: Dienstag, den 27. November 2007, 13:54:11 Uhr
Betreff: Re: AW: Does tc-prio really work as advertised?

Joerg Pommnitz wrote:
>  > It works fine here, I'm guessing that Jörg is using an old kernel
>  > version that had a bug in prio classification without filters.
> 
> This is 2.6.20.21, from 17-Oct-2007. 


Yes, that version is broken. I think it was fixed in 2.6.22 or 2.6.23.






      Jetzt Mails schnell in einem Vorschaufenster überfliegen. Dies und viel mehr bietet das neue Yahoo! Mail - www.yahoo.de/mail

Re: Does tc-prio really work as advertised?

[Patrick McHardy]

Joerg Pommnitz wrote:
> Great :-(. I went over the ChangeLogs for later kernel versions, but
> couldn't find anything.
> 
> I'll try to come up with a working .config for the most current kernel
> that works on my hardware and test again.
> 
> Do you have a pointer to the fix so that I could try to patch a 2.6.20
> kernel?

git-log net/sched/sch_prio.c:

commit dbaaa07a60edf71670b484a65e14400ab40c84f7
Author: Lucas Nussbaum <lucas.nussbaum@imag.fr>
Date:   Thu Aug 30 22:35:46 2007 -0700

     [NET_SCHED] sch_prio.c: remove duplicate call of tc_classify()

and

commit bdba91ec70fb5ccbdeb1c7068319adc6ea9e1a7d
Author: Patrick McHardy <kaber@trash.net>
Date:   Mon Jul 30 17:07:14 2007 -0700

     [NET_SCHED]: Fix prio/ingress classification logic error

Feel free to send patches against the still maintained -stable versions.

Re: Does tc-prio really work as advertised?

[Jarek Poplawski]

On Tue, Nov 27, 2007 at 05:00:55AM -0800, Joerg Pommnitz wrote:
...
> The OLSR traffic was obviously generated locally (not forwarded).
> You claim, that the TOS value for locally generated traffic does
> not influence its priority?

I wouldn't say "I claim": I've found something like this 'long time
ago' (enough to forget), and didn't need this information since
then. Now, I see you have 'nearby' problem, so it's only a hint.
After your first message I think you should be able to check this
by yourself. But, if there are any problems with this, you find my
idea wrong or not valid anymore, I'm interested in knowing about
this.

Cheers,
Jarek P.

Re: Does tc-prio really work as advertised?

[Michael Blizek]

Hi!

On 05:00 Tue 27 Nov     , Joerg Pommnitz wrote:
> > So, are you still sure you've tested such a case?
> 
> Well, the problem that triggered my investigation was
> that the OLSR daemon (www.olsr.org) calculates the quality
> of a link according to the packet loss for LQ HELLO packets
> (UDP broadcast packets). To prevent other traffic from
> interfering with the LQ calculation, olsrd sends the HELLO
> packets with a TOS value of 0x10 (minimize delay). This
> should give them the highest priority.
> 
> What I saw was a degrading Link quality with more user traffic
> over a link. The LQ fell so far that olsrd judged the other host
> unreachable and deleted the routing entry. The user traffic in
> question was iperf (TOS value 0x00).
> --  
> Regards and thanks for taking an interest
> 
>  
>        Joerg

There is another possible cause of this problem.

In WLANs all packets are acked or retransmitted. This is done to
prevent packet loss due to noise and collisions. But this is not
done with broadcast packets.

I can't tell you if this can be enough to trigger the bevaviour
you experienced. Is the signal barely receivable?

Michi
-- 
programing a layer 3+4 network protocol for mesh networks
see http://michaelblizek.homelinux.net