Asserting ECN from userspace?

Asserting ECN from userspace?

[David Täht]

No sooner had I noted (with pleasure) the kernel's new ability to
correctly set the dscp bits on IPv6 TCP streams without messing with the
negotiated ECN status, that I found several use cases where being able
to assert ECN from userspace (for either ipv4, or ipv6) would be useful.

1) Applications such as bittorrent (transmission, etc) that are much
more aware of their overall environment could assert ECN on their UDP
streams to indicate congestion.

2) Test tools. It would be nice to be able, from userspace, to easily
diagnose if ECN was working on a stream, end to end, and being able to
set and receive the ECN bits on a less algorithmic basis (ie, not wedged
deep within a kernel aqm such as RED or SFB)

3) Web Proxies. A web proxy could note when it was experiencing
congestion on one side of the proxied connection (or another) and signal
the other side to slow down.

Ah, ECN, we hardly know ye.

as for item 1 I'm hard pressed to think of a case where setting the ECN
bits on udp streams would introduce a security problem.

As for 2, can live without.

As for 3... perhaps a grantable network capability? A proxy could
acquire privs to twiddle those bits before dropping root privs.

That begs the question of how to see those bits in the first place. OOB
data?

And twiddling them, on a per stream basis, for a single packet, would
seem to require something more robust than setsockopt/getsockopt
(although that would work for udp streams)

Re: Asserting ECN from userspace?

[Andi Kleen]

David Täht <dave.taht@gmail.com> writes:
>
> And twiddling them, on a per stream basis, for a single packet, would
> seem to require something more robust than setsockopt/getsockopt
> (although that would work for udp streams)

With netfilter nf_queue you can construct a rule that passes packets
through user space and reinjects them.

I would suggest to just use that to modify the ECN bits.

I'm sure with reasonable google skills you can find some examples
how to do this on the web.

-Andi


-- 
ak@linux.intel.com -- Speaking for myself only

Re: Asserting ECN from userspace?

[Eric Dumazet]

Le mardi 04 octobre 2011 à 23:18 -0700, David Täht a écrit :
> No sooner had I noted (with pleasure) the kernel's new ability to
> correctly set the dscp bits on IPv6 TCP streams without messing with the
> negotiated ECN status, that I found several use cases where being able
> to assert ECN from userspace (for either ipv4, or ipv6) would be useful.
> 
> 1) Applications such as bittorrent (transmission, etc) that are much
> more aware of their overall environment could assert ECN on their UDP
> streams to indicate congestion.
> 
> 2) Test tools. It would be nice to be able, from userspace, to easily
> diagnose if ECN was working on a stream, end to end, and being able to
> set and receive the ECN bits on a less algorithmic basis (ie, not wedged
> deep within a kernel aqm such as RED or SFB)
> 
> 3) Web Proxies. A web proxy could note when it was experiencing
> congestion on one side of the proxied connection (or another) and signal
> the other side to slow down.
> 
> Ah, ECN, we hardly know ye.
> 
> as for item 1 I'm hard pressed to think of a case where setting the ECN
> bits on udp streams would introduce a security problem.
> 
> As for 2, can live without.
> 
> As for 3... perhaps a grantable network capability? A proxy could
> acquire privs to twiddle those bits before dropping root privs.
> 
> That begs the question of how to see those bits in the first place. OOB
> data?
> 
> And twiddling them, on a per stream basis, for a single packet, would
> seem to require something more robust than setsockopt/getsockopt
> (although that would work for udp streams)
> 

For UDP, its really easy to set ECT(0) or ECT(1) in your outgoing
frames, and test as well ECT(0),ECT(1),ECN in incoming frames.

For the sending part:
int tos = 0x2; /* ECT(1) */
setsockopt(fd, IP_PROTOIP, IP_TOS, &tos, sizeof(tos));

To be able to get the TOS value :

int on = 1;
setsockopt(fd, IP_PROTOIP, IP_RECVTOS, &on, sizeof(on));

RECVTOS (since Linux 2.2)
     If  enabled the IP_TOS ancillary message is passed with incoming packets.
     It contains a byte which specifies the Type of Service/Precedence
     field of the packet header.  Expects a boolean integer flag.

Re: Asserting ECN from userspace?

[Rick Jones]

On 10/04/2011 11:18 PM, David Täht wrote:
>
> No sooner had I noted (with pleasure) the kernel's new ability to
> correctly set the dscp bits on IPv6 TCP streams without messing with the
> negotiated ECN status, that I found several use cases where being able
> to assert ECN from userspace (for either ipv4, or ipv6) would be useful.
> ...
> 3) Web Proxies. A web proxy could note when it was experiencing
> congestion on one side of the proxied connection (or another) and signal
> the other side to slow down.
> ...
> As for 3... perhaps a grantable network capability? A proxy could
> acquire privs to twiddle those bits before dropping root privs.

For 3, couldn't/shouldn't the proxy simply stop draining the appropriate 
socket buffer to cause TCP's existing flow control to slow down that side?

rick jones

Re: Asserting ECN from userspace?

[Juliusz Chroboczek]

Dave,

I'm not sure what you are getting at.  ECN is designed for routers, not
for end-points.  Assering ECN congestion-experienced at the sender will
cause the sender to react to the congestion indication after a whole RTT
(after the ECN-echo is received).  For end-to-end flow control, it is
both simpler and more efficient to reduce the sending rate immediately,
without going over the network.

There's a good reason why we're careful to distinguish congestion
control (router-to-endpoint) and flow control (endpoint-to-endpoint).
The latter is much easier.

> 1) Applications such as bittorrent (transmission, etc) that are much
> more aware of their overall environment could assert ECN on their UDP
> streams to indicate congestion.

The sender can react to congestion by simply reducing the sending rate.
The receiver can react to congestion by pipelining fewer chunk requests
to the sender.

> 3) Web Proxies. A web proxy could note when it was experiencing
> congestion on one side of the proxied connection (or another) and signal
> the other side to slow down.

It can cause the other side to slow down by simply stopping reading,
thus causing the normal TCP flow control (not congestion control) to
kick in.

What would be useful, on the other hand, would be the ability to set the
ECN enabled codepoint on UDP packets, and have some means to reliably
check whether the Congestion-Experienced codepoint has been set by some
intermediate router.  But that's different from what you appear to be
suggesting.

-- Juliusz

Re: Asserting ECN from userspace?

[David Täht]

[-- Attachment #1: Type: text/plain, Size: 3499 bytes --]


My original attempt at sending this was blocked due to the ip address
embedded in the url I'd included.... resending...

On 10/13/2011 01:30 PM, Juliusz Chroboczek wrote:
> Dave,
>
> I'm not sure what you are getting at.  ECN is designed for routers, not
> for end-points.

Which happens to be what I'm primarily working on at the moment.
Secondly, in mesh networks, all machines are routers.

>   Assering ECN congestion-experienced at the sender will
> cause the sender to react to the congestion indication after a whole RTT
> (after the ECN-echo is received).  For end-to-end flow control, it is
> both simpler and more efficient to reduce the sending rate immediately,
> without going over the network.
>
> There's a good reason why we're careful to distinguish congestion
> control (router-to-endpoint) and flow control (endpoint-to-endpoint).
> The latter is much easier.
My message was overly broad in scope and I failed to distinguish between 'asserting ECN from userspace', and 'indicating a flow was ECN capable'


>> 1) Applications such as bittorrent (transmission, etc) that are much
>> more aware of their overall environment could assert ECN on their UDP
>> streams to indicate congestion.
> The sender can react to congestion by simply reducing the sending rate.
> The receiver can react to congestion by pipelining fewer chunk requests
> to the sender.
>

And what I meant here, was more 'indicate a flow is ecn capable' than
'assert ECN', so routers in the middle can do better signalling.

...although I can still imagine circumstances where asserting ECN makes
sense at the endpoint. See for example:

http://en.wikipedia.org/wiki/Data_Center_TCP

>> 3) Web Proxies. A web proxy could note when it was experiencing
>> congestion on one side of the proxied connection (or another) and signal
>> the other side to slow down.
> It can cause the other side to slow down by simply stopping reading,
> thus causing the normal TCP flow control (not congestion control) to
> kick in.

Yes, but in this case a TCP flow will continue to scale the window and
thus sending rate until those buffers fill. Signalling ECN sooner would
allow a middlebox/web proxy/split tcp session that is blocking on send
to throttle the overall rate from the other side at the speed it is
actually capable of retransmitting.

> What would be useful, on the other hand, would be the ability to set the
> ECN enabled codepoint on UDP packets, 

Agreed.

> and have some means to reliably
> check whether the Congestion-Experienced codepoint has been set by some
> intermediate router. 
>From the previous discussion on this thread it looks as though the core
capabilities exist, if not application code... which looks simple to
play with, and then apply to the AQM work ongoing.

>  But that's different from what you appear to be
> suggesting.
>

You make my thoughts clearer, as always.

Also, as we discussed elsewhere, setting the ECN capable bit on streams
such as VOIP has a packet
preservation-through-a-single-congested-endpoint feature that may well
be useful.

Completely as a side note, I've been looking over the shortest queue
first algorithm for AQM, which is quite promising in and of itself,
without qos or signalling needed at all.

My first attempt at sending this mail failed due to the embedded ip
address for the paper in the google search - if you search for this, you
can find the paper I'm referring to:

  Self-Prioritization of Audio and Video Traffic

> -- Juliusz


-- 
Dave Täht


[-- Attachment #2: dave_taht.vcf --]
[-- Type: text/x-vcard, Size: 204 bytes --]

begin:vcard
fn;quoted-printable:Dave T=C3=A4ht
n;quoted-printable:T=C3=A4ht;Dave
email;internet:dave.taht@gmail.com
tel;home:1-239-829-5608
tel;cell:0638645374
x-mozilla-html:FALSE
version:2.1
end:vcard