[RFC] IP_RECVERRC

[RFC] IP_RECVERRC

[jamal]

It seems there are a lot of dumbass apps (latest i have found is iperf
when analyzing batching results) out there whose performance is affected
if they dont set IP_RECVERR. 
If you set that option though you end up getting all these skbs back
to the app which i see as unnecessary work if i am uninterested. I would
like to not do a recvmsg to find out what those messages are - rather
just receiving any errors back.
My proposal is to add another option which is mutually exclusive with
IP_RECVERR that allows for errors only.
Can i get a ye before i implement?

Andi, CCing you because i think you may be the person who added
IP_RECVERR (at least the first time i heard it a few years back was from
you).

cheers,
jamal

Re: [RFC] IP_RECVERRC

[Andi Kleen]

On Fri, Aug 10, 2007 at 09:35:12AM -0400, jamal wrote:
> 
> It seems there are a lot of dumbass apps (latest i have found is iperf
> when analyzing batching results) out there whose performance is affected
> if they dont set IP_RECVERR. 

Affected in what way? 

> If you set that option though you end up getting all these skbs back

IP_RECVERR just delivers ICMPs. Do you mean those with "all these skbs" ?

> to the app which i see as unnecessary work if i am uninterested. I would
> like to not do a recvmsg to find out what those messages are - rather
> just receiving any errors back.
> My proposal is to add another option which is mutually exclusive with
> IP_RECVERR that allows for errors only.

What ICMPs do you not want? 

-Andi

Re: [RFC] IP_RECVERRC

[jamal]

On Fri, 2007-10-08 at 16:02 +0200, Andi Kleen wrote:
> On Fri, Aug 10, 2007 at 09:35:12AM -0400, jamal wrote:

> 
> Affected in what way? 
> 

They dont get errors back and they just keep sending even in the
presence of errors - take a look at ip_push_pending_frames. I have been
struggling initially to unconditionally mod that part, but it may break
certain apps expectations.

> > If you set that option though you end up getting all these skbs back
> 
> IP_RECVERR just delivers ICMPs. Do you mean those with "all these skbs" ?

yes - sorry thats what i meant; the app has to do a recvmsg to grab them

> > to the app which i see as unnecessary work if i am uninterested. I would
> > like to not do a recvmsg to find out what those messages are - rather
> > just receiving any errors back.
> > My proposal is to add another option which is mutually exclusive with
> > IP_RECVERR that allows for errors only.
> 
> What ICMPs do you not want? 

All.

cheers,
jamal

Re: [RFC] IP_RECVERRC

[Andi Kleen]

On Fri, Aug 10, 2007 at 10:30:12AM -0400, jamal wrote:
> On Fri, 2007-10-08 at 16:02 +0200, Andi Kleen wrote:
> > On Fri, Aug 10, 2007 at 09:35:12AM -0400, jamal wrote:
> 
> > 
> > Affected in what way? 
> > 
> 
> They dont get errors back and they just keep sending even in the
> presence of errors - take a look at ip_push_pending_frames. I have been
> struggling initially to unconditionally mod that part, but it may break
> certain apps expectations.

Are we talking about TCP or UDP/RAW?

The reason why IP_RECVERR was invented was that for non connected 
DGRAM sockets there was no way to figure out for which target host
the error was. So they were dropped. With IP_RECVERR you can get them
together with the original target address.

If you connect the DGRAM socket and don't set IP_RECVERR you will just get 
errnos -- that might be what you want.

If you're talking about TCP: IP_RECVERR never made too much sense here.

Normally TCP only reports errors after a major timeout.
At some point Alexey added an extension that if IP_RECVERR is set
you won't need to wait on the timeout, but get the error immediately
delivered using MSG_ERRQUEUE. That actually breaks TCP because it 
is not resistent to shifting routes etc. anymore so it's a pretty bad idea.

If for TCP you just want the error delivery without MSG_ERRQUEUE
you could just decrease the max timeouts. Then major timeouts
will happen sooner and you'll get the errnos sooner too.
Of course TCP will be far more unreliable than it is. But it might
be ok for some few applications who really know what they're doing
(but more likely it is just programmer hybris who think
they know all about the networks the app will ever be deployed on) 

Anyways, it might make sense to set these timeouts per socket (right now
it is only global sysctl), but on the other hand that would bloat
the already bloated tcp_sock even more.

-Andi

Re: [RFC] IP_RECVERRC

[jamal]

On Fri, 2007-10-08 at 18:26 +0200, Andi Kleen wrote:
> Are we talking about TCP or UDP/RAW?

Both. initially datagram.

I think you gave me something to think about - let me go back to the
drawing table. I like the per socket timer idea; however, if i can solve
from the app (or write my own app which is fine) then i will rather go
that path.
Thanks for taking the time Andi.

cheers,
jamal