DROP still returns -EPERM to userspace in OUTPUT chain

DROP still returns -EPERM to userspace in OUTPUT chain

[Jan Engelhardt]

Hi,


once again, irc snatched this report:

|2009-05-20T20:56 < Wintre:#Netfilter>
|
|Specifically, when I add a DROP rule to the local firewall, send(2)
|starts getting EPERM. The netfilter core code includes
|nf_hook_slow(), which says:
|
| /* Returns 1 if okfn() needs to be executed by the caller,
|  * -EPERM for NF_DROP, 0 otherwise. */
|
|So, this seems kind of crazy to me. I always thought drop was
|supposed to be silent, and changing the return value of send(2),
|well. Bad. Anybody got a link to a discussion of this issue? Or is it
|just a plain old bug?

I agree with the user here. For now, one had to make use of the
“STEAL” target [1] to get the real silent drop behavior for the
OUTPUT chain. Surely that is not the ideal thing either.
Requesting comments from NF maintainers.

[1] http://xtables-addons.git.sourceforge.net/git/gitweb.cgi?p=xtables-addons;a=blob;hb=HEAD;f=extensions/xt_STEAL.c
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: DROP still returns -EPERM to userspace in OUTPUT chain

[Pablo Neira Ayuso]

Jan Engelhardt wrote:
> Hi,
> 
> 
> once again, irc snatched this report:
> 
> |2009-05-20T20:56 < Wintre:#Netfilter>
> |
> |Specifically, when I add a DROP rule to the local firewall, send(2)
> |starts getting EPERM. The netfilter core code includes
> |nf_hook_slow(), which says:
> |
> | /* Returns 1 if okfn() needs to be executed by the caller,
> |  * -EPERM for NF_DROP, 0 otherwise. */
> |
> |So, this seems kind of crazy to me. I always thought drop was
> |supposed to be silent, and changing the return value of send(2),
> |well. Bad. Anybody got a link to a discussion of this issue? Or is it
> |just a plain old bug?
> 
> I agree with the user here. For now, one had to make use of the
> “STEAL” target [1] to get the real silent drop behavior for the
> OUTPUT chain. Surely that is not the ideal thing either.
> Requesting comments from NF maintainers.

I'm curious, what application would need to ignore that error? Returning
-EPERM seems to me quite sane to note that the kernel is explicit (via
iptables, for example) not allowing permission to send().

-- 
"Los honestos son inadaptados sociales" -- Les Luthiers
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: DROP still returns -EPERM to userspace in OUTPUT chain

[Jan Engelhardt]

On Saturday 2009-05-23 12:47, Pablo Neira Ayuso wrote:
>> 
>> once again, irc snatched this report:
>> 
>> |2009-05-20T20:56 < Wintre:#Netfilter>
>> |
>> |Specifically, when I add a DROP rule to the local firewall, send(2)
>> |starts getting EPERM. The netfilter core code includes
>> |nf_hook_slow(), which says:
>> |
>> | /* Returns 1 if okfn() needs to be executed by the caller,
>> |  * -EPERM for NF_DROP, 0 otherwise. */
>> |
>> |So, this seems kind of crazy to me. I always thought drop was
>> |supposed to be silent, and changing the return value of send(2),
>> |well. Bad. Anybody got a link to a discussion of this issue? Or is it
>> |just a plain old bug?
>> 
>> I agree with the user here. For now, one had to make use of the
>> “STEAL” target [1] to get the real silent drop behavior for the
>> OUTPUT chain. Surely that is not the ideal thing either.
>> Requesting comments from NF maintainers.
>
>I'm curious, what application would need to ignore that error?

Well, the problem is that applications do not check for all
potential errors, just whether there was one or not:

 while (there_is_data)
     if (send(fd, &data[i], len) < 0) {
         perror("the connection broke, or something we can't fix");
         break;
     }

Sure it feels a bit like the reliance on ext3 update intervals :-)

> Returning
>-EPERM seems to me quite sane to note that the kernel is explicit (via
>iptables, for example) not allowing permission to send().

Yeah but DROP is perceived by users to be "silently ignore it",
while the "you don't have permission" is REJECT's job.
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: DROP still returns -EPERM to userspace in OUTPUT chain

[Pablo Neira Ayuso]

Jan Engelhardt wrote:
> On Saturday 2009-05-23 12:47, Pablo Neira Ayuso wrote:
>>> once again, irc snatched this report:
>>>
>>> |2009-05-20T20:56 < Wintre:#Netfilter>
>>> |
>>> |Specifically, when I add a DROP rule to the local firewall, send(2)
>>> |starts getting EPERM. The netfilter core code includes
>>> |nf_hook_slow(), which says:
>>> |
>>> | /* Returns 1 if okfn() needs to be executed by the caller,
>>> |  * -EPERM for NF_DROP, 0 otherwise. */
>>> |
>>> |So, this seems kind of crazy to me. I always thought drop was
>>> |supposed to be silent, and changing the return value of send(2),
>>> |well. Bad. Anybody got a link to a discussion of this issue? Or is it
>>> |just a plain old bug?
>>>
>>> I agree with the user here. For now, one had to make use of the
>>> “STEAL” target [1] to get the real silent drop behavior for the
>>> OUTPUT chain. Surely that is not the ideal thing either.
>>> Requesting comments from NF maintainers.
>> I'm curious, what application would need to ignore that error?
> 
> Well, the problem is that applications do not check for all
> potential errors, just whether there was one or not:
> 
>  while (there_is_data)
>      if (send(fd, &data[i], len) < 0) {
>          perror("the connection broke, or something we can't fix");
>          break;
>      }

Hm, I see. Then I think that people should fix their applications instead.

> Sure it feels a bit like the reliance on ext3 update intervals :-)
> 
>> Returning
>> -EPERM seems to me quite sane to note that the kernel is explicit (via
>> iptables, for example) not allowing permission to send().
> 
> Yeah but DROP is perceived by users to be "silently ignore it",
> while the "you don't have permission" is REJECT's job.

But the DROP and REJECT behaviours refer to the packet logic, ie. with
DROP nothing is done, with REJECT we send some explicit packet (like an
ICMP administratively prohibited). That still applies to user-space.

Reporting -EPERM seems to me a good practise to report user-space
applications that the kernel is explicit dropping the packet. Otherwise,
while diagnosing problems, people cannot be sure where the packet has
been lost.

I'm more like in favour of forcing people to fix their applications,
they are doing a broken error handling.

-- 
"Los honestos son inadaptados sociales" -- Les Luthiers
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: DROP still returns -EPERM to userspace in OUTPUT chain

[Jan Engelhardt]

On Saturday 2009-05-23 13:43, Pablo Neira Ayuso wrote:
>>> Returning
>>> -EPERM seems to me quite sane to note that the kernel is explicit (via
>>> iptables, for example) not allowing permission to send().
>> 
>> Yeah but DROP is perceived by users to be "silently ignore it",
>> while the "you don't have permission" is REJECT's job.
>
>But the DROP and REJECT behaviours refer to the packet logic, ie. with
>DROP nothing is done, with REJECT we send some explicit packet (like an
>ICMP administratively prohibited). That still applies to user-space.

-EPERM is an "administrative prohibited" for userspace, just like a
returned ICMP packet. Here, functions overlap.

>Reporting -EPERM seems to me a good practise to report user-space
>applications that the kernel is explicit dropping the packet. Otherwise,
>while diagnosing problems, people cannot be sure where the packet has
>been lost.

Then again, people might be using -m limit -j DROP to simulate actual
packet loss, for whatever scientific interests they currently have.
So just wanting to know - are people supposed to use xt_STEAL instead
if they really want it silently dropped?

>I'm more like in favour of forcing people to fix their applications,
>they are doing a broken error handling.

Think UDP :)

Re: DROP still returns -EPERM to userspace in OUTPUT chain

[Pablo Neira Ayuso]

Jan Engelhardt wrote:
> On Saturday 2009-05-23 13:43, Pablo Neira Ayuso wrote:
>>>> Returning
>>>> -EPERM seems to me quite sane to note that the kernel is explicit (via
>>>> iptables, for example) not allowing permission to send().
>>> Yeah but DROP is perceived by users to be "silently ignore it",
>>> while the "you don't have permission" is REJECT's job.
>> But the DROP and REJECT behaviours refer to the packet logic, ie. with
>> DROP nothing is done, with REJECT we send some explicit packet (like an
>> ICMP administratively prohibited). That still applies to user-space.
> 
> -EPERM is an "administrative prohibited" for userspace, just like a
> returned ICMP packet. Here, functions overlap.

Indeed, I forgot about that case.

>> Reporting -EPERM seems to me a good practise to report user-space
>> applications that the kernel is explicit dropping the packet. Otherwise,
>> while diagnosing problems, people cannot be sure where the packet has
>> been lost.
> 
> Then again, people might be using -m limit -j DROP to simulate actual
> packet loss, for whatever scientific interests they currently have.

For scientific purposes, like packet omission emulation, better to use
netem [1].

> So just wanting to know - are people supposed to use xt_STEAL instead
> if they really want it silently dropped?

Well, I still would like to know any application that can benefit from
this, apart from broken applications.

[1] http://www.linuxfoundation.org/en/Net:Netem

-- 
"Los honestos son inadaptados sociales" -- Les Luthiers

Re: DROP still returns -EPERM to userspace in OUTPUT chain

[Patrick McHardy]

Pablo Neira Ayuso wrote:
> Jan Engelhardt wrote:
>> So just wanting to know - are people supposed to use xt_STEAL instead
>> if they really want it silently dropped?
> 
> Well, I still would like to know any application that can benefit from
> this, apart from broken applications.

I'd suggest to encode an errno code in the verdict and return that
one. Currently we're not able to propagate f.i. -EHOSTUNREACH from
ip_route_me_harder() and always return -EPERM. This could then be
used to make the errno code configurable for DROP, similar to
unreachable routes.

Re: DROP still returns -EPERM to userspace in OUTPUT chain

[Mike Acar]

I'd like to re-open this discussion. I apologize for not responding
sooner; I've been a bit busy. I'm also not subscribed to
netfilter-devel, so this message may bounce from there.

On Sat, May 23, 2009 at 6:20 AM, Jan Engelhardt<jengelh@medozas.de> wrote:

> Then again, people might be using -m limit -j DROP to simulate actual
> packet loss, for whatever scientific interests they currently have.

Which is precisely what happened: I was using DROP to simulate packet
loss to test timeout handling in a program. The program in question
does handle errors, but that wasn't the code path I wanted to
exercise. I wasn't aware of netem, but DROP would be all I needed in
this case (if it didn't return -EPERM).

In my former life as a sysadmin, it never occurred to me to interpret
DROP as "administratively prohibited"; that is what REJECT is for. I
interpreted DROP as "drop the packet silently, without any response",
which I think is the intuitive interpretation. An ICMP reply to a
remote machine is a response, and changing the return value of a
system call is also a response; neither is desirable.

The current behavior produces different results on local and remote
machines - programs on the remote machine time out, while programs on
the local machine get an error. I think this inconsistency - or
asymmetry - is undesirable.

What happens when adding an INPUT DROP rule for a protocol and port
bound for a socket where a daemon is listening? If we apply this
interpretation consistently, then when the rule is added, those
listen() calls should be interrupted and return -EPERM. I don't think
that's desirable behavior either - I think the kernel should drop the
packets when they arrive, and the listening daemon should never know
it happened.

Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> Reporting -EPERM seems to me a good practise to report user-space
> applications that the kernel is explicit dropping the packet. Otherwise,
> while diagnosing problems, people cannot be sure where the packet has
> been lost.

I don't agree. In fact, the current behavior makes this worse, because
the -EPERM behavior is unexpected (I think the interpretation of DROP
as silent is very common) and inconsistent (different things happen if
you're dropping remotely versus locally) - so it's not like you can
forget that you must check both end's firewalling rules when you're
diagnosing a problem.

-- Mike Acar

Re: DROP still returns -EPERM to userspace in OUTPUT chain

[Patrick McHardy]

Mike Acar wrote:
> I'd like to re-open this discussion. I apologize for not responding
> sooner; I've been a bit busy. I'm also not subscribed to
> netfilter-devel, so this message may bounce from there.
> 
> On Sat, May 23, 2009 at 6:20 AM, Jan Engelhardt<jengelh@medozas.de> wrote:
> 
>> Then again, people might be using -m limit -j DROP to simulate actual
>> packet loss, for whatever scientific interests they currently have.
> 
> Which is precisely what happened: I was using DROP to simulate packet
> loss to test timeout handling in a program. The program in question
> does handle errors, but that wasn't the code path I wanted to
> exercise. I wasn't aware of netem, but DROP would be all I needed in
> this case (if it didn't return -EPERM).
> 
> In my former life as a sysadmin, it never occurred to me to interpret
> DROP as "administratively prohibited"; that is what REJECT is for. I
> interpreted DROP as "drop the packet silently, without any response",
> which I think is the intuitive interpretation. An ICMP reply to a
> remote machine is a response, and changing the return value of a
> system call is also a response; neither is desirable.
> 
> The current behavior produces different results on local and remote
> machines - programs on the remote machine time out, while programs on
> the local machine get an error. I think this inconsistency - or
> asymmetry - is undesirable.
> 
> What happens when adding an INPUT DROP rule for a protocol and port
> bound for a socket where a daemon is listening? If we apply this
> interpretation consistently, then when the rule is added, those
> listen() calls should be interrupted and return -EPERM. I don't think
> that's desirable behavior either - I think the kernel should drop the
> packets when they arrive, and the listening daemon should never know
> it happened.
> 
> Pablo Neira Ayuso <pablo@netfilter.org> wrote:
>> Reporting -EPERM seems to me a good practise to report user-space
>> applications that the kernel is explicit dropping the packet. Otherwise,
>> while diagnosing problems, people cannot be sure where the packet has
>> been lost.
> 
> I don't agree. In fact, the current behavior makes this worse, because
> the -EPERM behavior is unexpected (I think the interpretation of DROP
> as silent is very common) and inconsistent (different things happen if
> you're dropping remotely versus locally) - so it's not like you can
> forget that you must check both end's firewalling rules when you're
> diagnosing a problem.

As I said, I'd gladly take a patch to a) propagate errno codes from
netfilter by encoding them in the verdict and b) make the exact
code configurable. The default needs to stay -EPERM however.