[LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[jamal]

For starters, i think you need a defintions sections. Look at:
http://www.ietf.org/internet-drafts/draft-ietf-diffserv-model-06.txt

(eg what is a shaper etc and how trhings are placed together). At least
that will ensure that you dont sday things like "Prio cant shape".

It is a good model but may be insufficient given Linux TCs
capabilities. Email me when unsure.

Some other things:
- In your comment "Do not confuse this classless simple qdisc with the
classful PRIO one!". This is misleading:
the default 3 band FIFO queue is conceptually the same as the
default prio qdisc (the priomaps are identical). 3 bands; same
prioritization schemes.
- You really need to fix ingress section:
 it works for both forwarding and packets coming in to local sockets.
More importantly, It takes advantages of _all_ filter schemes
available for TC as well as the policing functionality (which sadly seemed
to have been replicated by someone in netfilter, wrongly if i may add ;->).
- You keep saying "reodering" -- dont know what that means. Reordering is
generally considered a Bad Thing(tm).

- your description of the "peakrate" (same in TBF as well as policing)

Well captured. It took ages to get this into peoples heads. This also
applies to CBQ.

- your description of "MTU"
Not very good description:
This is just what it literally says; maximum transmit unit;
A packet larger than this will be dropped. Default is 2K. For ethernet,
MTUs of 1500 bytes, this is fine; however, you should put a cautionary
statement here in regards to people having MTUs smaller than 2K (example
the lo device); they might find that all their packets greater than 2K
being dropped.


More later if dont get distracted.

cheers,
jamal



_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

[LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[bert hubert]

On Sat, Dec 08, 2001 at 03:43:05PM -0500, jamal wrote:

> For starters, i think you need a defintions sections. Look at:
> http://www.ietf.org/internet-drafts/draft-ietf-diffserv-model-06.txt
> 
> (eg what is a shaper etc and how trhings are placed together). At least
> that will ensure that you dont sday things like "Prio cant shape".

I see that now :-) The right wording appears to be that a Prio is a
Work-conserving non-policing shaper.

> It is a good model but may be insufficient given Linux TCs
> capabilities. Email me when unsure.

Will do.

> Some other things:
> - In your comment "Do not confuse this classless simple qdisc with the
> classful PRIO one!". This is misleading:
> the default 3 band FIFO queue is conceptually the same as the
> default prio qdisc (the priomaps are identical). 3 bands; same
> prioritization schemes.

New wording:
Do not confuse this classless simple qdisc with the classful PRIO one!
Although they have a lot in common, the PRIO queue can contain different
classes, whereas pfifo_fast has hardcoded FIFO bands.


> - You really need to fix ingress section:
>  it works for both forwarding and packets coming in to local sockets.
> More importantly, It takes advantages of _all_ filter schemes
> available for TC as well as the policing functionality (which sadly seemed
> to have been replicated by someone in netfilter, wrongly if i may add ;->).

Yeah, it's broken, it counts packets, not bytes. It does praise Alexey
though :-)

Ok, new ingress description:

The ingress qdisc is a strange animal in that is not used to send packets
out to the network adaptor. Instead, it allows you to apply tc filters to
packets coming in over the interface, regardless of whether they have a
local destination or are to be forwarded.

As the tc filters contain a full Token Bucket Filter implementation, and are
also able to match on the kernel flow estimator, there is a lot of
functionality available. This effectively allows you to police incoming
traffic, before it even enters the IP stack.

Parameters & usage

The ingress qdisc itself does not require any parameters. It differs from
other qdiscs in that it does not occupy the root of a device. Attach it like
this:

# tc qdisc add dev eth0 ingress

This allows you to have other, sending, qdiscs on your device besides the
ingress qdisc.

For a contrived example how the ingress qdisc could be used, see the
Cookbook.

> - You keep saying "reodering" -- dont know what that means. Reordering is
> generally considered a Bad Thing(tm).

Well. That is what it comes down to - it reorders packets. It does not
reorder them within the same tcp/ip session, or at least, we hope so. In
other words, it delays certain packets while it doesn't delay others. How
would you suggest wording this? 

> - your description of "MTU"
> Not very good description:
> This is just what it literally says; maximum transmit unit;
> A packet larger than this will be dropped. Default is 2K. For ethernet,
> MTUs of 1500 bytes, this is fine; however, you should put a cautionary
> statement here in regards to people having MTUs smaller than 2K (example
> the lo device); they might find that all their packets greater than 2K
> being dropped.

Sure? From linux/net/sched/sch_tbf.c: 

                 toks = PSCHED_TDIFF_SAFE(now, q->t_c, q->buffer, 0);

                if (q->P_tab) {
                        ptoks = toks + q->ptokens;
                        if (ptoks > (long)q->mtu)
                                ptoks = q->mtu;
                        ptoks -= L2T_P(q, skb->len);
                }

Sure looks like the mtu, measured in tokens, is the size of the second
bucket?

> More later if dont get distracted.

Thanks.

-- 
http://www.PowerDNS.com          Versatile DNS Software & Services
Trilab                                 The Technology People
Netherlabs BV / Rent-a-Nerd.nl           - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

[LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[jamal]

On Sat, 8 Dec 2001, bert hubert wrote:

> On Sat, Dec 08, 2001 at 03:43:05PM -0500, jamal wrote:
>
> > For starters, i think you need a defintions sections. Look at:
> > http://www.ietf.org/internet-drafts/draft-ietf-diffserv-model-06.txt
> >
> > (eg what is a shaper etc and how trhings are placed together). At least
> > that will ensure that you dont sday things like "Prio cant shape".
>
> I see that now :-) The right wording appears to be that a Prio is a
> Work-conserving non-policing shaper.
>

work conserving is right. non-policing is wrong. Policing is related to
filters. Shaping is related to schedulers. Prio is a scheduler.
Shaping results in non-work conserving schemes. You can attacha TBF which
shaping inside a Prio qdisc. That would add non-workconserving-ness to it.

> > It is a good model but may be insufficient given Linux TCs
> > capabilities. Email me when unsure.
>
> Will do.
>

Basically dont take it for the gospel.

> > Some other things:
> > - In your comment "Do not confuse this classless simple qdisc with the
> > classful PRIO one!". This is misleading:
> > the default 3 band FIFO queue is conceptually the same as the
> > default prio qdisc (the priomaps are identical). 3 bands; same
> > prioritization schemes.
>
> New wording:
> Do not confuse this classless simple qdisc with the classful PRIO one!
> Although they have a lot in common, the PRIO queue can contain different
> classes, whereas pfifo_fast has hardcoded FIFO bands.

I am not sure if i like the wording: A class is the result of a
class-ification. Both pfifo_fast and PRIO have builtin class-ifiers.
Essentially if you treated default prio qdisc and pfifo_fast as black
boxes, there is _no_ difference. Dont look at the code, think larger
picture.

>
>
> > - You really need to fix ingress section:
> >  it works for both forwarding and packets coming in to local sockets.
> > More importantly, It takes advantages of _all_ filter schemes
> > available for TC as well as the policing functionality (which sadly seemed
> > to have been replicated by someone in netfilter, wrongly if i may add ;->).
>
> Yeah, it's broken, it counts packets, not bytes. It does praise Alexey
> though :-)
>

I think the main problem i found with it is that it used a single token
bucket.

> Ok, new ingress description:
>
> The ingress qdisc is a strange animal in that is not used to send packets
> out to the network adaptor. Instead, it allows you to apply tc filters to
> packets coming in over the interface, regardless of whether they have a
> local destination or are to be forwarded.
>
> As the tc filters contain a full Token Bucket Filter implementation, and are
> also able to match on the kernel flow estimator, there is a lot of
> functionality available. This effectively allows you to police incoming
> traffic, before it even enters the IP stack.
>

You can have two qdiscs per device, ingress and egress
Please look at the router model i described earlier. The two hooks are
very clearly described there.
Ingress qdisc is work conserving only by design; whereas egress qdiscs
could be non-work conserving.

> Parameters & usage
>
> The ingress qdisc itself does not require any parameters. It differs from
> other qdiscs in that it does not occupy the root of a device. Attach it like
> this:
>
> # tc qdisc add dev eth0 ingress
>
> This allows you to have other, sending, qdiscs on your device besides the
> ingress qdisc.
>

Again, look at the definitiions of eg/ingress. You need to have this
diagram drawn:

http://www.davin.ottawa.on.ca/ols/img9.htm
in your document.

> For a contrived example how the ingress qdisc could be used, see the
> Cookbook.
>
> > - You keep saying "reodering" -- dont know what that means. Reordering is
> > generally considered a Bad Thing(tm).
>
> Well. That is what it comes down to - it reorders packets. It does not
> reorder them within the same tcp/ip session, or at least, we hope so. In
> other words, it delays certain packets while it doesn't delay others. How
> would you suggest wording this?

Look at the model draft then lets talk again.

>
> > - your description of "MTU"
> > Not very good description:
> > This is just what it literally says; maximum transmit unit;
> > A packet larger than this will be dropped. Default is 2K. For ethernet,
> > MTUs of 1500 bytes, this is fine; however, you should put a cautionary
> > statement here in regards to people having MTUs smaller than 2K (example
> > the lo device); they might find that all their packets greater than 2K
> > being dropped.
>
> Sure?

100% sure. Try a little experiment then look at the code again.

cheers,
jamal


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

[LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[bert hubert]

On Sat, Dec 08, 2001 at 04:56:04PM -0500, jamal wrote:
> > I see that now :-) The right wording appears to be that a Prio is a
> > Work-conserving non-policing shaper.
> 
> work conserving is right. non-policing is wrong. Policing is related to
> filters. Shaping is related to schedulers. Prio is a scheduler.

Ok, so 'work-conserving' encodes the fact that it will never delay packets?

> Shaping results in non-work conserving schemes. You can attacha TBF which
> shaping inside a Prio qdisc. That would add non-workconserving-ness to it.

Ok, but pfifo_fast for example will always be work conserving.

> > New wording:
> > Do not confuse this classless simple qdisc with the classful PRIO one!
> > Although they have a lot in common, the PRIO queue can contain different
> > classes, whereas pfifo_fast has hardcoded FIFO bands.
> 
> I am not sure if i like the wording: A class is the result of a
> class-ification. Both pfifo_fast and PRIO have builtin class-ifiers.
> Essentially if you treated default prio qdisc and pfifo_fast as black
> boxes, there is _no_ difference. Dont look at the code, think larger
> picture.

Well, I aim for the user. For the user, the big picture may be identical but
the use is quite different. I don't want to get email 'I tried to add a
qdisc to pfifo_fast and it didn't work!'. 

New wording:
 Do not confuse this classless simple qdisc with the classful PRIO one!
 Although they behave similarly, pfifo_fast is classless and you cannot add
 other qdiscs to it with the tc command.

I think this covers what you mean and what I want.

> You can have two qdiscs per device, ingress and egress
> Please look at the router model i described earlier. The two hooks are
> very clearly described there.
> Ingress qdisc is work conserving only by design; whereas egress qdiscs
> could be non-work conserving.

True. The ingress qdisc isn't really dequeued.

> Again, look at the definitiions of eg/ingress. You need to have this
> diagram drawn:
> 
> http://www.davin.ottawa.on.ca/ols/img9.htm
> in your document.

It's good, will need to asciify it however.

> > other words, it delays certain packets while it doesn't delay others. How
> > would you suggest wording this?
> 
> Look at the model draft then lets talk again.

Ok. Rewording of the HOWTO will have to wait on a glossary/definition
section anyhow.

> > > the lo device); they might find that all their packets greater than 2K
> > > being dropped.
> >
> > Sure?
> 
> 100% sure. Try a little experiment then look at the code again.

I will, but I bet you 5 euros that I'm right :-) We are talking about the
TBF, aren't we?

Regards,

bert

-- 
http://www.PowerDNS.com          Versatile DNS Software & Services
Trilab                                 The Technology People
Netherlabs BV / Rent-a-Nerd.nl           - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

[LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[jamal]

On Sun, 9 Dec 2001, bert hubert wrote:

> On Sat, Dec 08, 2001 at 04:56:04PM -0500, jamal wrote:

> Ok. Rewording of the HOWTO will have to wait on a glossary/definition
> section anyhow.
>

You need that termininology section.

> > > > the lo device); they might find that all their packets greater than 2K
> > > > being dropped.
> > >
> > > Sure?
> >
> > 100% sure. Try a little experiment then look at the code again.
>
> I will, but I bet you 5 euros that I'm right :-) We are talking about the
> TBF, aren't we?
>

We are talking about a dual token bucket in relation to filter policing.
It would probably cost you more than 5 Euros charges to send 5 Euros to me
in .ca ;-> So just keep it.
How about i raise my guarantee to 200%? ;->

cheers,
jamal



_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

[LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[jamal]

On Sun, 9 Dec 2001, bert hubert wrote:

>
> I will, but I bet you 5 euros that I'm right :-) We are talking about the
> TBF, aren't we?
>

The trick is that TBF delays packets and the policer drops packet when
exceeding their profile.

cheers,
jamal


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Henrik Nordstrom]

On Sunday 09 December 2001 00.08, bert hubert wrote:
>
> True. The ingress qdisc isn't really dequeued.

One confusing thing is that the tc command supports one to add another qdisc 
to ingress. Have not yet figured out if this actually can be used for 
anything.

   tc qdisc add dev eth0 ingress tbf ....
   (or any other qdisc)

is happily accepted but afaict does not seem to have any function..

Regards
Henrik Nordström

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

[LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[bert hubert]

On Sat, Dec 08, 2001 at 06:30:47PM -0500, jamal wrote:
> 
> 
> On Sun, 9 Dec 2001, bert hubert wrote:
> 
> >
> > I will, but I bet you 5 euros that I'm right :-) We are talking about the
> > TBF, aren't we?
> >
> 
> The trick is that TBF delays packets and the policer drops packet when
> exceeding their profile.

Well, we're both right to some extent. The policer drops packets which are
larger than the configured MTU, but it also dimensions the second bucket to
be of size mtu! 

From net/sched/police.c (trimmed a bit, added some comments):
int tcf_police(struct sk_buff *skb, struct tcf_police *p)
{
        psched_time_t now;
        long toks,ptoks = 0;

        spin_lock(&p->lock);

        p->stats.bytes += skb->len;
        p->stats.packets++;

        if (skb->len <= p->mtu) {
                if (p->R_tab = NULL) {
                        spin_unlock(&p->lock);
                        return p->result;
                }

                PSCHED_GET_TIME(now);

                toks = PSCHED_TDIFF_SAFE(now, p->t_c, p->burst, 0);
		// tokens that arrived since last invocation

                if (p->P_tab) {
                        ptoks = toks + p->ptoks;
                        if (ptoks > (long)L2T_P(p, p->mtu))
                                ptoks = (long)L2T_P(p, p->mtu); 
			// cap total available ptokens to mtu
                        ptoks -= L2T_P(p, skb->len);
                        // deduct packet size
                }
                toks += p->toks; 
                if (toks > (long)p->burst)
                        toks = p->burst;
                // cap regular number of tokens

                toks -= L2T(p, skb->len);
                // deduct packet size

                if ((toks|ptoks) >= 0) { // send if both are positive
                        p->t_c = now;
                        p->toks = toks;  // do accounting
                        p->ptoks = ptoks;
                        spin_unlock(&p->lock);
                        return p->result;
                }
        }

        p->stats.overlimits++;
        spin_unlock(&p->lock);
        return p->action;
}

It's a draw? :-)

Regards,

bert

-- 
http://www.PowerDNS.com          Versatile DNS Software & Services
Trilab                                 The Technology People
Netherlabs BV / Rent-a-Nerd.nl           - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

[LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[jamal]

On Sun, 9 Dec 2001, bert hubert wrote:

> On Sat, Dec 08, 2001 at 06:30:47PM -0500, jamal wrote:
> >
> >
> > On Sun, 9 Dec 2001, bert hubert wrote:
> >
> > >
> > > I will, but I bet you 5 euros that I'm right :-) We are talking about the
> > > TBF, aren't we?
> > >
> >
> > The trick is that TBF delays packets and the policer drops packet when
> > exceeding their profile.
>
> Well, we're both right to some extent. The policer drops packets which are
> larger than the configured MTU, but it also dimensions the second bucket to
> be of size mtu!
>

[..]

> It's a draw? :-)

Actually you get the 5 euro. My comments were in regards to the
_policer_ but they were made to comment on the _shaper_ in that specific
section of the document.
So i suppose when you explain the policer my comments would apply.
[The rest of the parameters apply fine]

cheers,
jamal


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[bert hubert]

On Sat, Dec 08, 2001 at 08:19:07PM -0500, jamal wrote:

> > It's a draw? :-)
> 
> Actually you get the 5 euro. My comments were in regards to the
> _policer_ but they were made to comment on the _shaper_ in that specific
> section of the document.

I hope to see you at OLS :-)

Regards,

bert

-- 
http://www.PowerDNS.com          Versatile DNS Software & Services
Trilab                                 The Technology People
Netherlabs BV / Rent-a-Nerd.nl           - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[jamal]

On Sun, 9 Dec 2001, bert hubert wrote:

> On Sat, Dec 08, 2001 at 08:19:07PM -0500, jamal wrote:
>
> > > It's a draw? :-)
> >
> > Actually you get the 5 euro. My comments were in regards to the
> > _policer_ but they were made to comment on the _shaper_ in that specific
> > section of the document.
>
> I hope to see you at OLS :-)

Ok, then you shall get your drink of choice worth 5 euros ;->

cheers,
jamal



_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[jamal]

On Sun, 9 Dec 2001, Henrik Nordstrom wrote:

> to ingress. Have not yet figured out if this actually can be used for
> anything.
>
>    tc qdisc add dev eth0 ingress tbf ....
>    (or any other qdisc)
>
> is happily accepted but afaict does not seem to have any function..

I cant seem to get it to work;
-------
[root@jzny tc]# ./tc qdisc add dev lo handle ffff: ingress tbf
RTNETLINK answers: Invalid argument
[root@jzny tc]# ./tc qdisc add dev lo handle ffff: ingress
[root@jzny tc]# tc -s qdisc
qdisc ingress ffff: dev lo
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
--------

What is the full command?
can you try tc -s qdisc after you add it ?

cheers,
jamal


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Henrik Nordstrom]

You need to specify the correct arguments to the qdisc you are adding.. I.e. 
the same command as adding that qdisc as the root qdisc, but replace root by 
ingress.

tc qdisc add dev eth0 ingress tbf rate 220kbit latency 50ms burst 1540

qdisc tbf ffff: dev eth0 rate 220Kbit burst 1407b lat 2147.5s 
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0) 

But as I said it does not seem to ever be used.

Regards
Henrik

On Sunday 09 December 2001 03.30, jamal wrote:
> On Sun, 9 Dec 2001, Henrik Nordstrom wrote:
> > to ingress. Have not yet figured out if this actually can be used for
> > anything.
> >
> >    tc qdisc add dev eth0 ingress tbf ....
> >    (or any other qdisc)
> >
> > is happily accepted but afaict does not seem to have any function..
>
> I cant seem to get it to work;
> -------
> [root@jzny tc]# ./tc qdisc add dev lo handle ffff: ingress tbf
> RTNETLINK answers: Invalid argument
> [root@jzny tc]# ./tc qdisc add dev lo handle ffff: ingress
> [root@jzny tc]# tc -s qdisc
> qdisc ingress ffff: dev lo
>  Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
> --------
>
> What is the full command?
> can you try tc -s qdisc after you add it ?
>
> cheers,
> jamal

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[bert hubert]

On Sun, Dec 09, 2001 at 12:38:21PM +0100, Henrik Nordstrom wrote:
> You need to specify the correct arguments to the qdisc you are adding.. I.e. 
> the same command as adding that qdisc as the root qdisc, but replace root by 
> ingress.
> 
> tc qdisc add dev eth0 ingress tbf rate 220kbit latency 50ms burst 1540
> 
> qdisc tbf ffff: dev eth0 rate 220Kbit burst 1407b lat 2147.5s 
>  Sent 0 bytes 0 pkts (dropped 0, overlimits 0) 
> 
> But as I said it does not seem to ever be used.

This appears to be a bug - which version of tc are you using? 

I think in fact tc just added an egress tbf - you can't specify *any* qdisc
as an ingress qdisc except for the bare one, like Jamal does below:

(jamal *wrote* the ingress qdisc, he should know)

> > I cant seem to get it to work;
> > -------
> > [root@jzny tc]# ./tc qdisc add dev lo handle ffff: ingress tbf
> > RTNETLINK answers: Invalid argument
> > [root@jzny tc]# ./tc qdisc add dev lo handle ffff: ingress
> > [root@jzny tc]# tc -s qdisc
> > qdisc ingress ffff: dev lo
> >  Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
> > --------
> >
> > What is the full command?
> > can you try tc -s qdisc after you add it ?

Regards,

bert

-- 
http://www.PowerDNS.com          Versatile DNS Software & Services
Trilab                                 The Technology People
Netherlabs BV / Rent-a-Nerd.nl           - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[jamal]

Hi,
This is definetely a bug; Is this with diffserv turned on in
iproute2/Config?
What version of iproute2 (not that it makes much of a difference)

cheers,
jamal

On Sun, 9 Dec 2001, Henrik Nordstrom wrote:

> You need to specify the correct arguments to the qdisc you are adding.. I.e.
> the same command as adding that qdisc as the root qdisc, but replace root by
> ingress.
>
> tc qdisc add dev eth0 ingress tbf rate 220kbit latency 50ms burst 1540
>
> qdisc tbf ffff: dev eth0 rate 220Kbit burst 1407b lat 2147.5s
>  Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
>
> But as I said it does not seem to ever be used.
>
> Regards
> Henrik
>
> On Sunday 09 December 2001 03.30, jamal wrote:
> > On Sun, 9 Dec 2001, Henrik Nordstrom wrote:
> > > to ingress. Have not yet figured out if this actually can be used for
> > > anything.
> > >
> > >    tc qdisc add dev eth0 ingress tbf ....
> > >    (or any other qdisc)
> > >
> > > is happily accepted but afaict does not seem to have any function..
> >
> > I cant seem to get it to work;
> > -------
> > [root@jzny tc]# ./tc qdisc add dev lo handle ffff: ingress tbf
> > RTNETLINK answers: Invalid argument
> > [root@jzny tc]# ./tc qdisc add dev lo handle ffff: ingress
> > [root@jzny tc]# tc -s qdisc
> > qdisc ingress ffff: dev lo
> >  Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
> > --------
> >
> > What is the full command?
> > can you try tc -s qdisc after you add it ?
> >
> > cheers,
> > jamal
>


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[jamal]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 100 bytes --]


Henrik,
Can you please try the attahed patch against iproute2-2.4.7-now-ss010824?


cheers,
jamal


[-- Attachment #2: Type: APPLICATION/octet-stream, Size: 817 bytes --]

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Henrik Nordstrom]

Something like 2.2.4 with diffserv enabled (would not allow ingress at all 
otherwise). To be precise the current RedHat rawhide package, slightly 
changed to enable diffserv support.

Regards
Henrik

On Sunday 09 December 2001 15.49, jamal wrote:
> Hi,
> This is definetely a bug; Is this with diffserv turned on in
> iproute2/Config?
> What version of iproute2 (not that it makes much of a difference)
>
> cheers,
> jamal

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Henrik Nordstrom]

On Sunday 09 December 2001 16.01, jamal wrote:
> Henrik,
> Can you please try the attahed patch against iproute2-2.4.7-now-ss010824?

Seems to make the tc userspace program to properly reject the arguments.


It is a bit sad that one cannot queue packets in ingress. Would be quite 
useful to make ingress shaping behave more sane than what can be acheived 
with the queueless filter police mechanism.


netfilter supports queueing/delaying of packets and then resume processing 
them at a later time using nf_reinject, so I think it should be possible to 
implement a ingress queue without too much effort.. but then the netfilter 
queueing seems to be very simplistic only supporting one queue per protocol 
family and this queueing interface is already used for queueing packets to 
userspace, so perhaps not as easy as I thought..

Queueing in netfilter works by

1. The queueing mechanism registers it's handler by calling 
nf_register_queue_handler. Only one queue handler per protocol family is 
supported.

2. On packets needed to be queued, return NF_QUEUE

3. When the queue handler is done with the packet, it calls nf_reinject with 
a new verdict.

4. If the packet was not dropped/stolen, netfilter processing continues at 
the next hook (not priority).

The queue handler gets the following information: skb, protocol family, nf 
hook number, and in/out devices.

Regards
Henrik

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[jamal]

On Sun, 9 Dec 2001, Henrik Nordstrom wrote:

> On Sunday 09 December 2001 16.01, jamal wrote:
> > Henrik,
> > Can you please try the attahed patch against iproute2-2.4.7-now-ss010824?
>
> Seems to make the tc userspace program to properly reject the arguments.
>

That was the intent; allowing it was harmless but gives a bad impression
of functionality. So i guess we could say the patch works.

>
> It is a bit sad that one cannot queue packets in ingress. Would be quite
> useful to make ingress shaping behave more sane than what can be acheived
> with the queueless filter police mechanism.
>

Look at the definition of work vs non-work conserving; This is design
intent. If you look at the datapath, it is totaly meaningless to put
queues at ingress, for routing when they are being queued on ingress as
well.

>
> netfilter supports queueing/delaying of packets and then resume processing
> them at a later time using nf_reinject, so I think it should be possible to
> implement a ingress queue without too much effort..

The implementation/extension is trivial. There is no need for it; I went
at great lengths with Martin/devik on this  Maybe he can help me here ;->

> but then the netfilter
> queueing seems to be very simplistic only supporting one queue per protocol
> family and this queueing interface is already used for queueing packets to
> userspace, so perhaps not as easy as I thought..
>

[..]

No, implementation is a non-issue. There is no need for it.
For 2.5 we might be able to have the ipqueue code use the power of TC. it
already talks netlink; i'll talk to some of the netfilter people. ipqueue
has some speacial need to grab packets; we provide much more sophisticated
mechanisms than Netfilter; so maybe there's a marriage possibility.

cheers,
jamal


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Michael T. Babcock]

On Sat, Dec 08, 2001 at 03:43:05PM -0500, jamal wrote:
> - You keep saying "reodering" -- dont know what that means. Reordering is
> generally considered a Bad Thing(tm).

Reordering happens on a mass scale (packets often go out in a different order
than they were received / generated) but not on a per-qdisc scale (packets
go out 'in order' within an SFQ queue or within a CBQ queue).  Its quite
obvious that fairness causes overall reordering of the available packets
because you sometimes with to pass along (for example) an SSH packet before
the 10 waiting FTP packets even though the latter got there first.
-- 
Michael T. Babcock
CTO, FibreSpeed Ltd.     (Hosting, Security, Consultation, Database, etc)
http://www.fibrespeed.net/~mbabcock/

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Michael T. Babcock]

On Sat, Dec 08, 2001 at 10:30:55PM +0100, bert hubert wrote:
> The ingress qdisc is a strange animal in that is not used to send packets
> out to the network adaptor. Instead, it allows you to apply tc filters to
> packets coming in over the interface, regardless of whether they have a
> local destination or are to be forwarded.

Opinions, opinions ... just the facts please.  My suggested paragraph:

The ingress qdisc allows the application of tc filters to the inbound 
packets on an interface instead of the outgoing ones.  This filtering 
is done to all incoming packets, whether destined for the local host or 
to be forwarded.
-- 
Michael T. Babcock
CTO, FibreSpeed Ltd.     (Hosting, Security, Consultation, Database, etc)
http://www.fibrespeed.net/~mbabcock/

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Cédric Rivard]

On Sunday 9 December 2001 22:41, jamal wrote:
> > It is a bit sad that one cannot queue packets in ingress. Would be quite
> > useful to make ingress shaping behave more sane than what can be acheived
> > with the queueless filter police mechanism.
>
> Look at the definition of work vs non-work conserving; This is design
> intent. If you look at the datapath, it is totaly meaningless to put
> queues at ingress, for routing when they are being queued on ingress as
> well.

Wouldn't it make sense to set a non-work conserving interface on ingress and 
a work conserving interface on egress ?
That would be handy to share bandwidth between outgoing packets through 
different interfaces.

Cedric

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

[LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Don Cohen]

I see that there are multiple interpretations of almost everything in
this exchange (and we both seem to be consistently choosing the wrong ones).

We all seem to agree now that the bandwidth specified for the class
should not necessarily be the bandwidth of the hardware device.
I was pointing out that the lartc howto seemed to indicate that it
should and that the new man page says that this parameter is different
from that, but does not tell us what it is!  I was not asking YOU to
tell me what it was supposed to be, but asking that the man page tell
us all.

Similarly for reordering, I misinterpreted
  (packets go out 'in order' within an SFQ queue or within a CBQ queue)
What you could have said in order for me to have interpreted this
correctly is something like
 packets go out of an sfq BUCKET (or SUBQUEUE) in FIFO order,
and
 within a CBQ CLASS packets are not reordered (other than by the qdisc
 connected to that class.
In both cases I interpreted the "queue" to mean the whole SFQ or CBQ queue.

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Martin Devera]

> > It is a bit sad that one cannot queue packets in ingress. Would be quite
> > useful to make ingress shaping behave more sane than what can be acheived
> > with the queueless filter police mechanism.
> >
> 
> Look at the definition of work vs non-work conserving; This is design
> intent. If you look at the datapath, it is totaly meaningless to put
> queues at ingress, for routing when they are being queued on ingress as
> well.

hehe, jamal did you remember long discussion we have had
about this (at diffserv list) ? :-)

> > netfilter supports queueing/delaying of packets and then resume processing
> > them at a later time using nf_reinject, so I think it should be possible to
> > implement a ingress queue without too much effort..
> 
> The implementation/extension is trivial. There is no need for it; I went
> at great lengths with Martin/devik on this  Maybe he can help me here ;->

yup I have not read whole message first :) So that you remember.
The conclusion was that only reason of queue at ingres might be
fact that existing queue stays here as indicator of flow's activity.

Definitely it would be helpful to create work conserving model
of CBQ (HTB :-)) which would drop packets instead to dequeue
them.

IMHO ingres queuing could be used as poor man's way how to reshape
(or priorize) traffic which can't be shaped at egress side (usualy
because of adminstrative boundaries). This need would vanish in
presence of such classfull work conserving CBQ.

Note that you can do some similar things with policers but you
can't do the same thinks as with CBQ - you can't set priorized
borrowing hierarchy up.

> For 2.5 we might be able to have the ipqueue code use the power of TC. it
> already talks netlink; i'll talk to some of the netfilter people. ipqueue
> has some speacial need to grab packets; we provide much more sophisticated
> mechanisms than Netfilter; so maybe there's a marriage possibility.

ipqueue !? what is it ? sounds good :)

regards, devik


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Henrik Nordstrom]

On Sunday 09 December 2001 22.41, jamal wrote:

> Look at the definition of work vs non-work conserving; This is design
> intent. If you look at the datapath, it is totaly meaningless to put
> queues at ingress, for routing when they are being queued on ingress as
> well.

(on egress as well I assume...)

True, but not all applications of shaping have the luxury of egress. For 
example, consider the not too uncommon example of a computer connected via 
100Mbps networking to a DSL modem, and you want to tune the use of the link 
without needing to introduce a router inbetween.

> The implementation/extension is trivial. There is no need for it; I went
> at great lengths with Martin/devik on this  Maybe he can help me here ;->

So do you have any argument why one should not be able to shape incoming 
local traffic to a station in a good manner without having a router do the 
shaping?
hh
Regards
Henrik

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Henrik Nordstrom]

On Monday 10 December 2001 09.38, Martin Devera wrote:

> Definitely it would be helpful to create work conserving model
> of CBQ (HTB :-)) which would drop packets instead to dequeue
> them.

Won't help me I think. I don't have the need of hierarchies or fancy 
prioritizations between different traffics or whatever, only to be able to 
slow down (with a delay) some specific traffic destined for the local TCP.

The filter police allows me to drop packets, but do not allow me to introduce 
delays.

TCP is generally too smart to be delayed proper by "randomly" dropped packets 
without any signs in RTT. Especially when the RTT is small.

> IMHO ingres queuing could be used as poor man's way how to reshape
> (or priorize) traffic which can't be shaped at egress side (usualy
> because of adminstrative boundaries). This need would vanish in
> presence of such classfull work conserving CBQ.

And such a administrative boundary is the one I am playing on. The boundary 
between a small customer and his ISP. The ISP obviously have the luxury of 
egress, but the customer does not on traffic received by him.

Exacly how would this need vanish?

Regards
Henrik

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Martin Devera]

> > Definitely it would be helpful to create work conserving model
> > of CBQ (HTB :-)) which would drop packets instead to dequeue
> > them.
> 
> Won't help me I think. I don't have the need of hierarchies or fancy 
> prioritizations between different traffics or whatever, only to be able to 
> slow down (with a delay) some specific traffic destined for the local TCP.
> 
> The filter police allows me to drop packets, but do not allow me to introduce 
> delays.
> 
> TCP is generally too smart to be delayed proper by "randomly" dropped packets 
> without any signs in RTT. Especially when the RTT is small.

Are you sure !? TCP slows down by half on every dropped
packet per congestion window AFAIK.
On other side it is often hard to slow TCP down by packet
delay as TCP will try to accomodate it by making MSS larger.
Am I right jamal ?

> 
> > IMHO ingres queuing could be used as poor man's way how to reshape
> > (or priorize) traffic which can't be shaped at egress side (usualy
> > because of adminstrative boundaries). This need would vanish in
> > presence of such classfull work conserving CBQ.
> 
> And such a administrative boundary is the one I am playing on. The boundary 
> between a small customer and his ISP. The ISP obviously have the luxury of 
> egress, but the customer does not on traffic received by him.
> 
> Exacly how would this need vanish?

as I said above, packet dropping works well (at least for me
in the same ISP scenario).
When you are queuing then delay only helps you to postpone 
burst of reply data to some less used time.
When bulk traffic persists, packet is droppes and TCP fallbacks
down.

devik


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Martin Devera]

> True, but not all applications of shaping have the luxury of egress. For 
> example, consider the not too uncommon example of a computer connected via 
> 100Mbps networking to a DSL modem, and you want to tune the use of the link 
> without needing to introduce a router inbetween.
> 
> > The implementation/extension is trivial. There is no need for it; I went
> > at great lengths with Martin/devik on this  Maybe he can help me here ;->
> 
> So do you have any argument why one should not be able to shape incoming 
> local traffic to a station in a good manner without having a router do the 
> shaping?

jamal would probably say here that it is nonsence to delay/queue packet
which already arived to your box :)
I still think that is COULD be helpful and much more clear
to be able to attach any qdisc at ingres.

Current "ingres" qdisc could remain and work in the same way but
additionaly when qdisc returns OK - I eat the packet - then net_bh
would try to dequeue it.
Then there would be no "special" qdisc like ingres - it would
be subset of regular qdisc.

The only real (implementation) problem I see is related to
netif_restart - we'd need rather tc subsys suplied callback
sch_restart(sch) which would do different thing for ingres
and egres. All other parts should be easy to implement.

On other note. I created so called IMQ device which solves
different (but similar) problem. It should be easy to extend
it to queue all incoming packets here too. Now only all 
outgoing ones go here.
IMQ is pretty simple device and is already used in production
environment by two people.

devik


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Henrik Nordstrom]

On Monday 10 December 2001 12.35, Martin Devera wrote:

> Are you sure !? TCP slows down by half on every dropped
> packet per congestion window AFAIK.
> On other side it is often hard to slow TCP down by packet
> delay as TCP will try to accomodate it by making MSS larger.
> Am I right jamal ?

You basically need both, or your packet drops will constantly be fighting 
retransmits as TCP is trying to recover. There will eventually be a balance, 
but not a too nice one.

> as I said above, packet dropping works well (at least for me
> in the same ISP scenario).
> When you are queuing then delay only helps you to postpone
> burst of reply data to some less used time.
> When bulk traffic persists, packet is droppes and TCP fallbacks
> down.

Sure, just dropping packets will work to some extent, but not by far as 
efficient in transmitted data as limited queue with drop on overflow 
(preferably smarter than FIFO+overflow if you need support more than one 
concurrent TCP session).

Regards
Henrik

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[bert hubert]

On Mon, Dec 10, 2001 at 10:59:38AM +0100, Henrik Nordstrom wrote:

> TCP is generally too smart to be delayed proper by "randomly" dropped packets 
> without any signs in RTT. Especially when the RTT is small.

Richard Stevens disagrees with you. 

> And such a administrative boundary is the one I am playing on. The boundary 
> between a small customer and his ISP. The ISP obviously have the luxury of 
> egress, but the customer does not on traffic received by him.
> 
> Exacly how would this need vanish?

You can turn ingress into egress by inserting another machine of course.
Ingress shaping, well, is weird if you have no concept of an 'ingress
queue'.

Regards,

bert

-- 
http://www.PowerDNS.com          Versatile DNS Software & Services
Trilab                                 The Technology People
Netherlabs BV / Rent-a-Nerd.nl           - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Henrik Nordstrom]

On Monday 10 December 2001 12.59, Martin Devera wrote:

> jamal would probably say here that it is nonsence to delay/queue packet
> which already arived to your box :)

In a station trying to shape the traffic sent to him it does by limiting the 
waste of retransmits. egress queues does not help then as there is no egress 
where to queue the packet.

To argue that it is nonsense to have a ingress queue for your own received 
packets is the same as to argue that it is nonsense to have a egress queue 
for routed packets. The packet dynamics are the same, only the application is 
slightly different.

To summary: ingress queues makes sense when you want to shape the traffic 
sent to you.

This is most obviously true when looking at a station, but also true to some 
extent in routers. To get the minds working on the router case consider the 
following router:

 ppp0	256 Kpbs link to ISP
 eth0	100 Mbps lan 1
 eth1	100 Mbps lan 2
 eth3	100 Mbps lan 3

And you want without needing to involve your ISP reasonably shape incoming 
traffic received on ppp0 to not use more than 64kbps for rsync in total.

Doing such a setup using ingress shaping is "trivial". Doing it using egress 
shaping on each of the ethernet interfaces is not.

Sure, the queues is only of use if the queue can be large enough to fit 
significant portions of your active TCP windows, but when they are they can 
significantly increase link efficiency by trading per packet latency.

Regards
Henrik

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Martin Devera]

> This is most obviously true when looking at a station, but also true to some 
> extent in routers. To get the minds working on the router case consider the 
> following router:
> 
>  ppp0	256 Kpbs link to ISP
>  eth0	100 Mbps lan 1
>  eth1	100 Mbps lan 2
>  eth3	100 Mbps lan 3
> 
> And you want without needing to involve your ISP reasonably shape incoming 
> traffic received on ppp0 to not use more than 64kbps for rsync in total.
> 
> Doing such a setup using ingress shaping is "trivial". Doing it using egress 
> shaping on each of the ethernet interfaces is not.

for this special case I designed IMQ :)


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[jamal]

On Mon, 10 Dec 2001, Henrik Nordstrom wrote:

> On Monday 10 December 2001 12.59, Martin Devera wrote:
>
> > jamal would probably say here that it is nonsence to delay/queue packet
> > which already arived to your box :)
>
> In a station trying to shape the traffic sent to him it does by limiting the
> waste of retransmits. egress queues does not help then as there is no egress
> where to queue the packet.
>
> To argue that it is nonsense to have a ingress queue for your own received
> packets is the same as to argue that it is nonsense to have a egress queue
> for routed packets. The packet dynamics are the same, only the application is
> slightly different.

No,
I would strongly suggest you run tests with dropped vs delayed TCP
packets. What you'll see is that even when you delay TCP packets retransmits
will happen. So this is a weak reason.
At least Martin and I agreed that the only reason youd need ingress is to
maintain the same TC semantics across ingress and egress;

As for the ipqueue folks there is a certain limitation with netlink at the
moment (hence the per-protocol family issue); so we might have to help
them queue packets in the kernel; pass only the headers to user space; let
them make a decision on the fate of the packet and some qdisc will act on
that decision. Now that is the hardway of doing things; the easy way is to
fix netlink.

Going back to hiding under paying work

cheers,
jamal




_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Martin Devera]

> > Are you sure !? TCP slows down by half on every dropped
> > packet per congestion window AFAIK.
> > On other side it is often hard to slow TCP down by packet
> > delay as TCP will try to accomodate it by making MSS larger.
> > Am I right jamal ?
> 
> You basically need both, or your packet drops will constantly be fighting 
> retransmits as TCP is trying to recover. There will eventually be a balance, 
> but not a too nice one.

I'd paste part of old jamal's email here:
----
Recall Mathis equation: TCP b/width = C*MSS/(RTT*sqrt(p))
where p is the drop probability and C is a constant which changes under
different conditions
----
Additionaly TCP tries to adapt window to RTT so that min(MSS,cwnd)/RTT
changes very slowly. Thus remaining part is C/sqrt(P). As you
see - it is hard to slow TCP down without dropping... 

If you will set up infinite queue and delay by 1s the TCP
will keep window increasing until drop occurs or reciever's
advertised window is hit.

I'd suggest you to read rfs2001.
devik



_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Jim Fleming]

----- Original Message ----- 
From: "bert hubert" <ahu@ds9a.nl>
To: "Henrik Nordstrom" <hno@marasystems.com>
Cc: "Martin Devera" <devik@cdi.cz>; "jamal" <hadi@cyberus.ca>; <lartc@mailman.ds9a.nl>
Sent: Monday, December 10, 2001 6:14 AM
Subject: Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages


> On Mon, Dec 10, 2001 at 10:59:38AM +0100, Henrik Nordstrom wrote:
> 
> > TCP is generally too smart to be delayed proper by "randomly" dropped packets 
> > without any signs in RTT. Especially when the RTT is small.
> 
> Richard Stevens disagrees with you. 
> 

I assume you mean the late Richard Stevens, who passed away at an early age
after helping to document the BSD Internet Protocol software in great detail.

BTW, while on the subject of TCP, Qs, etc. It seems rather odd that people
engrossed in protocols, would not consider the long history of making sure that
layers are preserved or at least considered. You may have heard of Layer 2
(Network) and Layer 3 (Transport). It would seem that much of the work on
Queues and the shaping of flows would most properly be located in Layer 3.
The additions we are making to Layer 2, focus on extending the addressing
and other functionality of Layer 2. It seems that Layer 2 will not have enough
bits to accommodate all of the ideas that people are pouring into it.

This may help...
http://www.dot-biz.com/IPv4/Tutorial/

Jim Fleming
http://www.IPv8.info
IPv16....One Better !!


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Michael T. Babcock]

On Mon, Dec 10, 2001 at 09:38:33AM +0100, Martin Devera wrote:
> IMHO ingres queuing could be used as poor man's way how to reshape
> (or priorize) traffic which can't be shaped at egress side (usualy
> because of adminstrative boundaries). This need would vanish in
> presence of such classfull work conserving CBQ.

Please fill me in -- how could it ever not be possible to shape
egress traffic?  Or are you refering to the egress traffic of the
upstream to the machine whose ingress side you wish to modify?
-- 
Michael T. Babcock
CTO, FibreSpeed Ltd.     (Hosting, Security, Consultation, Database, etc)
http://www.fibrespeed.net/~mbabcock/

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Michael T. Babcock]

On Mon, Dec 10, 2001 at 09:41:59AM +0100, Henrik Nordstrom wrote:
> example, consider the not too uncommon example of a computer connected via 
> 100Mbps networking to a DSL modem, and you want to tune the use of the link 
> without needing to introduce a router inbetween.

Assuming there is only one computer (and therefore no need for the router),
why would you want something other than a work-conserving ingress policy?
Drop certain packets, allow everything else ...

I can almost see your point if we're discussing very slow computers, but
in that case the qdisc's would slow it down more -- could you please fill
me in on your assumptions here?
-- 
Michael T. Babcock
CTO, FibreSpeed Ltd.     (Hosting, Security, Consultation, Database, etc)
http://www.fibrespeed.net/~mbabcock/

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Gerry Creager N5JXS]

bert hubert wrote:
> 
> On Mon, Dec 10, 2001 at 10:59:38AM +0100, Henrik Nordstrom wrote:
> 
> > TCP is generally too smart to be delayed proper by "randomly" dropped packets
> > without any signs in RTT. Especially when the RTT is small.
> 
> Richard Stevens disagrees with you.
> 
> > And such a administrative boundary is the one I am playing on. The boundary
> > between a small customer and his ISP. The ISP obviously have the luxury of
> > egress, but the customer does not on traffic received by him.
> >
> > Exacly how would this need vanish?
> 
> You can turn ingress into egress by inserting another machine of course.
> Ingress shaping, well, is weird if you have no concept of an 'ingress
> queue'.

Once you start working with tagging for DiffServ, you find that an
ingress queue is a valuable idea.  From our perspective here it is a
differentiator in looking at some of the "big iron" from the likes of
Juniper, Anritsu, Marconi, Cisco and Alcatel.

Specifically, we're looking at priority queueing for management of
various services:  VoIP, streaming video (unicast and multicast), H.323,
etc.  Ingress queueing provides us an opportunity to tag and shape
coming into the router rather than simply shaping on egress.  Our campus
requires (geographic considerations) 7 internal routers before we come
to the edge.  We have to shape on ingress at the first one, then
maintain the marking and policies throughout the network.
--
Gerry Creager -- gerry@cs.tamu.edu
Network Engineering
Academy for Advanced Telecommunications and Learning Technologies		
Texas A&M University	979.458.4020  (Phone) -- 979.847.8578  (Fax)

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Michael T. Babcock]

On Mon, Dec 10, 2001 at 01:00:41PM +0100, Henrik Nordstrom wrote:
> Sure, just dropping packets will work to some extent, but not by far as 
> efficient in transmitted data as limited queue with drop on overflow 
> (preferably smarter than FIFO+overflow if you need support more than one 
> concurrent TCP session).

This is why RED was created -- some of the research papers on RED show
how it will balance out TCP traffic very well if tuned properly, but
it isn't always perfectly straightforward.
-- 
Michael T. Babcock
CTO, FibreSpeed Ltd.     (Hosting, Security, Consultation, Database, etc)
http://www.fibrespeed.net/~mbabcock/

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Martin Devera]

> > IMHO ingres queuing could be used as poor man's way how to reshape
> > (or priorize) traffic which can't be shaped at egress side (usualy
> > because of adminstrative boundaries). This need would vanish in
> > presence of such classfull work conserving CBQ.
> 
> Please fill me in -- how could it ever not be possible to shape
> egress traffic?  Or are you refering to the egress traffic of the
> upstream to the machine whose ingress side you wish to modify?

exactly.


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages

[Jim Fleming]

----- Original Message ----- 
From: "Gerry Creager N5JXS" <gerry@cs.tamu.edu>
To: "bert hubert" <ahu@ds9a.nl>
Cc: "Henrik Nordstrom" <hno@marasystems.com>; "Martin Devera" <devik@cdi.cz>; "jamal" <hadi@cyberus.ca>; <lartc@mailman.ds9a.nl>
Sent: Monday, December 10, 2001 7:56 AM
Subject: Re: [LARTC] Re: further CBQ/tc documentation ds9a.nl/lartc/manpages


> bert hubert wrote:
> > 
> > On Mon, Dec 10, 2001 at 10:59:38AM +0100, Henrik Nordstrom wrote:
> > 
> > > TCP is generally too smart to be delayed proper by "randomly" dropped packets
> > > without any signs in RTT. Especially when the RTT is small.
> > 
> > Richard Stevens disagrees with you.
> > 
> > > And such a administrative boundary is the one I am playing on. The boundary
> > > between a small customer and his ISP. The ISP obviously have the luxury of
> > > egress, but the customer does not on traffic received by him.
> > >
> > > Exacly how would this need vanish?
> > 
> > You can turn ingress into egress by inserting another machine of course.
> > Ingress shaping, well, is weird if you have no concept of an 'ingress
> > queue'.
> 
> Once you start working with tagging for DiffServ, you find that an
> ingress queue is a valuable idea.  From our perspective here it is a
> differentiator in looking at some of the "big iron" from the likes of
> Juniper, Anritsu, Marconi, Cisco and Alcatel.
> 
> Specifically, we're looking at priority queueing for management of
> various services:  VoIP, streaming video (unicast and multicast), H.323,
> etc.  Ingress queueing provides us an opportunity to tag and shape
> coming into the router rather than simply shaping on egress.  Our campus
> requires (geographic considerations) 7 internal routers before we come
> to the edge.  We have to shape on ingress at the first one, then
> maintain the marking and policies throughout the network.
> --
> Gerry Creager -- gerry@cs.tamu.edu
> Network Engineering
> Academy for Advanced Telecommunications and Learning Technologies 
> Texas A&M University 979.458.4020  (Phone) -- 979.847.8578  (Fax)
> 


"We have to shape on ingress at the first one, then
maintain the marking and policies throughout the network."

It sounds like you need RIFRAF Routing.
RIFRAF - Remote Identification Field Random Action Filter


Jim Fleming
http://www.IPv8.info
IPv16....One Better !!



_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/