Configure ICMP error source address

Configure ICMP error source address

[Robert Sander]

[-- Attachment #1: Type: text/plain, Size: 1232 bytes --]

Hi,

It is possible to change the source address of ICMP error messages
generated by the kernel via
/proc/sys/net/ipv4/icmp_errors_use_inbound_ifaddr. This is currently the
only way to influence the source address as ICMP errors do not travel
through the NAT table (for obvious reasons).

We have the situation that our routers use RFC1918 addresses on their
transfer networks (which should be quite common nowadays to save on
public IPv4 addresses). ICMP errors are generated with RFC1918 source
addresses and therefor never reach the original sender.

Every router has its public IP address bound to dev lo to be reachable
even if any one interface is down. Routing protocols assure that.

Is it a good idea to develop a kernel patch that makes it possible to
select the first IPv4 address on dev lo with scope global as the source
address for ICMP errors? Would that do any harm to the Internet at large?

Regards
-- 
Robert Sander
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin

http://www.heinlein-support.de

Tel: 030 / 405051-43
Fax: 030 / 405051-19

Zwangsangaben lt. §35a GmbHG:
HRB 93818 B / Amtsgericht Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Configure ICMP error source address

[prmarino1]

Don't put a public address on a lo device use a dummy eth interface instead‎. Any IP address and it's subnet assigned to a lo device is  marked as a marcian address and the traffic is dropped if it tries to leave the lo device.

I know that there is som old documentation out there (for example quagga's documentation) that says you can do it ‎but it's been wrong since the 2.4 version off the kernel. 
Linux treats the lo device differently that what routers call a loopback device. The dummy driver is the linux equivalent of what routers call a loopback device.
  Original Message  
From: Robert Sander
Sent: Friday, January 8, 2016 04:32
To: netfilter@vger.kernel.org; netdev@vger.kernel.org
Subject: Configure ICMP error source address

Hi,

It is possible to change the source address of ICMP error messages
generated by the kernel via
/proc/sys/net/ipv4/icmp_errors_use_inbound_ifaddr. This is currently the
only way to influence the source address as ICMP errors do not travel
through the NAT table (for obvious reasons).

We have the situation that our routers use RFC1918 addresses on their
transfer networks (which should be quite common nowadays to save on
public IPv4 addresses). ICMP errors are generated with RFC1918 source
addresses and therefor never reach the original sender.

Every router has its public IP address bound to dev lo to be reachable
even if any one interface is down. Routing protocols assure that.

Is it a good idea to develop a kernel patch that makes it possible to
select the first IPv4 address on dev lo with scope global as the source
address for ICMP errors? Would that do any harm to the Internet at large?

Regards
-- 
Robert Sander
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin

http://www.heinlein-support.de

Tel: 030 / 405051-43
Fax: 030 / 405051-19

Zwangsangaben lt. §35a GmbHG:
HRB 93818 B / Amtsgericht Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin

Re: Configure ICMP error source address

[Hannes Frederic Sowa]

On 08.01.2016 16:24, prmarino1@gmail.com wrote:
> Don't put a public address on a lo device use a dummy eth interface instead‎. Any IP address and it's subnet assigned to a lo device is  marked as a marcian address and the traffic is dropped if it tries to leave the lo device.

O_o

> I know that there is som old documentation out there (for example quagga's documentation) that says you can do it ‎but it's been wrong since the 2.4 version off the kernel.
> Linux treats the lo device differently that what routers call a loopback device. The dummy driver is the linux equivalent of what routers call a loopback device.

What you write seems odd to me, we don't treat lo devices differently to 
dummy devices in respect if you bind a public ip address on it.

Bye,
Hannes

Re: Configure ICMP error source address

[prmarino1]

 The reason why is in the kernel documentation distributed with the source code.
‎Its not all immediately obvious why and where the kernel does this unless you were involved in the original debates about it. The problems with binding a public ip address to a loopback address are more recently is partialy selinux but mostly of it is because of how routing changed when iproute2 was introduced in the 2.2 kernel (if I remember the versions correctly it was so long ago). The reason is security, many processes bind to the loopback interface for IPC and just grab the first address on the first lo device they find, some of the developers of the applications weren't even aware they were doing it because in some cases it was C libraries several layers deep doing it because odd loopback addresses were common on UNIX ( I remember an AIX admin back in the early 90's telling me I should always use addresses like 127.1.0.2 on the loopback because it was valid made it harder for script kiddies to attack processes ). This was fine when additional addresses would need additional alias on the interface for ifconfig (lo:1 for example) but with iproute2 you could now assign multiple ip addresses to an interface without creating an alias (commonly seen on the keepalived mailing list "why doesn't my VIP show up in ifconfig?")
So there was a decision made to declare all subnets assigned to the loopback interface as unroutable to prevent a lot of very common security holes that were being seen in the wild.

That said you can make it work but you have to do a lot of things you should never do on a firewall like turn off selinux.‎ Or you can simply use the dummy driver and bind your public IP's to that and the problems magically go away. And it makes a lot of sense why else would there be a need for the dummy network driver?


  Original Message  
From: Hannes Frederic Sowa
Sent: Friday, January 8, 2016 11:11
To: prmarino1@gmail.com; Robert Sander; netfilter@vger.kernel.org; netdev@vger.kernel.org
Subject: Re: Configure ICMP error source address

On 08.01.2016 16:24, prmarino1@gmail.com wrote:
> Don't put a public address on a lo device use a dummy eth interface instead‎. Any IP address and it's subnet assigned to a lo device is marked as a marcian address and the traffic is dropped if it tries to leave the lo device.

O_o

> I know that there is som old documentation out there (for example quagga's documentation) that says you can do it ‎but it's been wrong since the 2.4 version off the kernel.
> Linux treats the lo device differently that what routers call a loopback device. The dummy driver is the linux equivalent of what routers call a loopback device.

What you write seems odd to me, we don't treat lo devices differently to 
dummy devices in respect if you bind a public ip address on it.

Bye,
Hannes

Re: Configure ICMP error source address

[Hannes Frederic Sowa]

On 09.01.2016 04:57, prmarino1@gmail.com wrote:
>   The reason why is in the kernel documentation distributed with the source code.
> ‎Its not all immediately obvious why and where the kernel does this
unless you were involved in the original debates about it. The problems
with binding a public ip address to a loopback address are more
recently is partialy selinux but mostly of it is because of how routing
changed when iproute2 was introduced in the 2.2 kernel (if I remember
the versions correctly it was so long ago). The reason is security,
many processes bind to the loopback interface for IPC and just grab the
first address on the first lo device they find, some of the developers
of the applications weren't even aware they were doing it because in
some cases it was C libraries several layers deep doing it because odd
loopback addresses were common on UNIX ( I remember an AIX admin back
in the early 90's telling me I should always use addresses like
127.1.0.2 on the loopback because it was valid made it harder for
script kiddies to attack processes ). This was fine when additional
addresses would need additiona
l alias on the interface for ifconfig (lo:1 for example) but with iproute2 you could now assign multiple ip addresses to an interface without creating an alias (commonly seen on the keepalived mailing list "why doesn't my VIP show up in ifconfig?")
> So there was a decision made to declare all subnets assigned to the loopback interface as unroutable to prevent a lot of very common security holes that were being seen in the wild.

selinux can only affect what happens between user space and kernel 
space, it cannot install policy what happens during forwarding if no 
user space is involved.

In general the kernel makes sure nothing leaves or enters the system 
with an address in the 127.0.0.0/8 subnet. Albeit we have a sysctl 
nowadays to lift this restriction, sysctl_route_localnet.

Besides that I don't see any differences, as I bind addresses to the 
loopback interface from time to time and it works.

One must just be careful if the kernel is switched from weak end mode 
into a stong end mode by enabling more strict arp filters, but this 
would also break setups with dummy interfaces.

> That said you can make it work but you have to do a lot of things you should never do on a firewall like turn off selinux.‎ Or you can simply use the dummy driver and bind your public IP's to that and the problems magically go away. And it makes a lot of sense why else would there be a need for the dummy network driver?

I would also use dummy interfaces in production systems, merely to split 
the statistics from dummy.

Thanks,
Hannes

Re: Configure ICMP error source address

[Robert Sander]

[-- Attachment #1: Type: text/plain, Size: 1414 bytes --]

Hi,

Am 09.01.2016 um 10:57 schrieb Hannes Frederic Sowa:
> 
> I would also use dummy interfaces in production systems, merely to split
> the statistics from dummy.

Thank you for discussing the merits of dummy interfaces. I will consider
your arguments. But unfortunately this did not answer my question.

Let me rephrase it:

Is it a good idea to set a specific global IPv4 address as source
address for outgoing ICMP error messages?

Would it be OK to create a /proc/sys/net/ipv4/icmp_errors_source where
you could write an arbitrary IPv4 address into? And that would get used
as the source address of ICMP errors?

My questions did contain the loopback interface as I first thought it a
good source of a globally routable IPv4 address (at least in our case).

Secound thought: Instead of writing an IPv4 address to
/proc/sys/net/ipv4/icmp_errors_source write an interface name to that
file and take the first global IPv4 address from that interface as
source for ICMP errors. Then you could create a dummy interface for that
use case, too.

Still: Is it a good idea to do so?

Regards
-- 
Robert Sander
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin

http://www.heinlein-support.de

Tel: 030 / 405051-43
Fax: 030 / 405051-19

Zwangsangaben lt. §35a GmbHG:
HRB 93818 B / Amtsgericht Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Configure ICMP error source address

[Pascal Hambourg]

Robert Sander a écrit :
> 
> Is it a good idea to set a specific global IPv4 address as source
> address for outgoing ICMP error messages?

Why only for ICMP error messages and not all outgoing packets sent to a
global destination address ?

Re: Configure ICMP error source address

[Hannes Frederic Sowa]

On 09.01.2016 17:41, Robert Sander wrote:
> Hi,
>
> Am 09.01.2016 um 10:57 schrieb Hannes Frederic Sowa:
>>
>> I would also use dummy interfaces in production systems, merely to split
>> the statistics from dummy.
>
> Thank you for discussing the merits of dummy interfaces. I will consider
> your arguments. But unfortunately this did not answer my question.

Yes, I know. :) I tried to answer it in the other reply.

> Let me rephrase it:
>
> Is it a good idea to set a specific global IPv4 address as source
> address for outgoing ICMP error messages?

Not sure if this generic enough.

So my idea was to have a specific routing table and ip rule you can 
install merely for selecting the source address of an icmp errors.

Not sure yet how complicated that is, it would require a match in the 
rule lookup logic to specifically use another routing table when the 
source address for an icmp packet is generated. We already supply the 
protocol in the flow4 information, maybe this can be used plus another 
input/flag in the flowi4 struct?

I can see situations were it is necessary to actually select the source 
address depending of the interface.

> Would it be OK to create a /proc/sys/net/ipv4/icmp_errors_source where
> you could write an arbitrary IPv4 address into? And that would get used
> as the source address of ICMP errors?
>
> My questions did contain the loopback interface as I first thought it a
> good source of a globally routable IPv4 address (at least in our case).
>
> Secound thought: Instead of writing an IPv4 address to
> /proc/sys/net/ipv4/icmp_errors_source write an interface name to that
> file and take the first global IPv4 address from that interface as
> source for ICMP errors. Then you could create a dummy interface for that
> use case, too.

I am not a fan of such implicit assumptions. I would prefer the direct 
specification of the source ip address over writing interface 
information to a procfs file.

> Still: Is it a good idea to do so?

I agree, there should be a solution for this as this is a common setup 
for BGP routers.

Bye,
Hannes

Re: Configure ICMP error source address

[Robert Sander]

[-- Attachment #1: Type: text/plain, Size: 2511 bytes --]

Am 10.01.2016 um 00:01 schrieb Hannes Frederic Sowa:

> I am not a fan of such implicit assumptions. I would prefer the direct
> specification of the source ip address over writing interface
> information to a procfs file.

I tried that but as I am not a seasoned kernel hacker introducing a new
sysctl including the validation of an IPv4 address was a bit too much.

Instead I created this patch (applicable against kernel version 3.2):

diff --git a/net/ipv4/icmp.c b/net/ipv4/icmp.c
index ab188ae..eba2071 100644
--- a/net/ipv4/icmp.c
+++ b/net/ipv4/icmp.c
@@ -567,7 +567,7 @@ void icmp_send(struct sk_buff *skb_in, int type, int code, __be32 info)
 
                rcu_read_lock();
                if (rt_is_input_route(rt) &&
-                   net->ipv4.sysctl_icmp_errors_use_inbound_ifaddr)
+                   net->ipv4.sysctl_icmp_errors_use_inbound_ifaddr == 1)
                        dev = dev_get_by_index_rcu(net, rt->rt_iif);
 
                if (dev)
@@ -577,6 +577,23 @@ void icmp_send(struct sk_buff *skb_in, int type, int code, __be32 info)
                rcu_read_unlock();
        }
 
+       /*
+        *      Set source in case of error reply
+        */
+
+       if (icmp_pointers[type].error && net->ipv4.sysctl_icmp_errors_use_inbound_ifaddr == 2) {
+               struct net_device *dev = NULL;
+               __be32 tmpaddr = 0;
+
+               rcu_read_lock();
+               dev = dev_get_by_name_rcu(net, "lo");
+               if (dev)
+                       tmpaddr = inet_select_addr(dev, 0, RT_SCOPE_UNIVERSE);
+                       if (tmpaddr)
+                               saddr = tmpaddr;
+               rcu_read_unlock();
+       }
+
        tos = icmp_pointers[type].error ? ((iph->tos & IPTOS_TOS_MASK) |
                                           IPTOS_PREC_INTERNETCONTROL) :
                                          iph->tos;


It currently works in my testbed (Debian wheezy based, therefor kernel 3.2).

Maybe there is someone more experienced with introducing new sysctl
files and handling strings in kernel space than me that is able to
pick up this idea and implement it properly.

Regards
-- 
Robert Sander
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin

http://www.heinlein-support.de

Tel: 030 / 405051-43
Fax: 030 / 405051-19

Zwangsangaben lt. §35a GmbHG: 
HRB 93818 B / Amtsgericht Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Configure ICMP error source address

[Hannes Frederic Sowa]

On 08.01.2016 10:31, Robert Sander wrote:
> Is it a good idea to develop a kernel patch that makes it possible to
> select the first IPv4 address on dev lo with scope global as the source
> address for ICMP errors? Would that do any harm to the Internet at large?

I think the way to go is to have a ip rule option in which the admin can 
add routes which get consulted only by the icmp source address 
determination logic. I can assume that some other installations use 
other interface number logic or multiple public addresses. This kind of 
lookup seems to allow all possible lookup scenarios.

Thoughts?

Hannes

Re: Configure ICMP error source address

[Robert Sander]

[-- Attachment #1: Type: text/plain, Size: 942 bytes --]

On 08.01.2016 10:31, Robert Sander wrote:

> We have the situation that our routers use RFC1918 addresses on their
> transfer networks (which should be quite common nowadays to save on
> public IPv4 addresses). ICMP errors are generated with RFC1918 source
> addresses and therefor never reach the original sender.

A follow-up to this request:

It is much easier to accomplish when settings routes with a source
address that is a public IP like this:

ip route add default via 10.10.10.1 src 192.0.2.24

ICMP errors will then use 192.0.2.24 as the source address.

No need for a kernel patch. Thanks for all the input.

Regards
-- 
Robert Sander
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin

http://www.heinlein-support.de

Tel: 030 / 405051-43
Fax: 030 / 405051-19

Zwangsangaben lt. §35a GmbHG:
HRB 93818 B / Amtsgericht Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]