Routing in case of GRE interface under a bridge

Routing in case of GRE interface under a bridge

[Nayan Gadre]

Hi,

I have a "l2gre0" and "eth0" interface under the bridge "br0". If a
packet comes to eth0 interface with a destination IP address say
10.10.10.1 which is not known on the Linux system, as there is no
route for 10.10.10.1, will the l2gre0 interface encapsulate this
packet and send it across the tunnel ?
The other endpoint is on a different Linux system with another l2gre0
interface having IP address 10.10.10.1

Thanks
N Gadre

Re: Routing in case of GRE interface under a bridge

[Ido Schimmel]

On Wed, Jun 28, 2023 at 07:06:45PM +0530, Nayan Gadre wrote:
> I have a "l2gre0" and "eth0" interface under the bridge "br0".

I assume "l2gre0" is a gretap, not ipgre.

> If a packet comes to eth0 interface with a destination IP address say
> 10.10.10.1 which is not known on the Linux system, as there is no
> route for 10.10.10.1, will the l2gre0 interface encapsulate this
> packet and send it across the tunnel ?

The bridge doesn't care about IP addresses when forwarding unicast
packets. Forwarding happens based on DMAC. Packet will be transmitted
through "l2gre0" if the bridge has a matching FDB entry for the DMAC
with "l2gre0" as the destination bridge port or if there is no FDB entry
at all, in which case the packet will be flooded.

One of the attributes of the GRE device is the remote address, which is
the encapsulating destination IP. Linux needs to have a route telling it
how to reach this destination address or the packet will be dropped.

> The other endpoint is on a different Linux system with another l2gre0
> interface having IP address 10.10.10.1
> 
> Thanks
> N Gadre
> 

Re: Routing in case of GRE interface under a bridge

[Nayan Gadre]

Yes, the l2gre0 on System A and System B is a gretap created using
following command

ip link add l2gre0 type gretap remote 192.168.0.10 local 192.168.0.103
            --> and vice versa on System B.

On system A, l2gre0 and eth0 are under a bridge br0. l2gre0 does not
have an IP address.
On system B, l2gre0 is independent but has IP address 10.10.10.1, and
a DHCP server is running on it providing IP to clients connected
through the tunnel.
                      System A
 |                               System B
                                            192.168.0.103            |
                     br0                         br1
    |                        eth0
l2gre0
           eth0           l2gre0            eth1                     |
                  192.168.0.10                        10.10.10.1

On system A:
/ # ip r
default via 192.168.0.10 dev br1.1
169.254.0.0/16 dev br1.1 proto kernel scope link src 169.254.32.107
192.168.0.0/24 dev br1.1 proto kernel scope link src 192.168.0.103

On system B:
ngadre@in01-7h4wrf3:~$ ip r
default via 10.110.234.254 dev eno1 proto dhcp metric 100
10.10.10.0/24 dev l2gre0 proto kernel scope link src 10.10.10.1
192.168.0.0/24 dev enp3s0 proto kernel scope link src 192.168.0.10

As we can see, on System B there is a route pointing at l2gre0.
However, there is no such route on System A. Yet the packet gets
encapsulated
A client connected to eth0 on System A sends packet with destination
10.10.10.1 (def gateway). So I am guessing l2gre0 receives this packet
since it gets flooded by br0 and even though System A not having a
route to 10.10.10.0/24 it will encapsulate. Is this the behavior in
case of a bridged tunnel interface ?.

On Thu, Jun 29, 2023 at 4:42 PM Ido Schimmel <idosch@idosch.org> wrote:
>
> On Wed, Jun 28, 2023 at 07:06:45PM +0530, Nayan Gadre wrote:
> > I have a "l2gre0" and "eth0" interface under the bridge "br0".
>
> I assume "l2gre0" is a gretap, not ipgre.
>
> > If a packet comes to eth0 interface with a destination IP address say
> > 10.10.10.1 which is not known on the Linux system, as there is no
> > route for 10.10.10.1, will the l2gre0 interface encapsulate this
> > packet and send it across the tunnel ?
>
> The bridge doesn't care about IP addresses when forwarding unicast
> packets. Forwarding happens based on DMAC. Packet will be transmitted
> through "l2gre0" if the bridge has a matching FDB entry for the DMAC
> with "l2gre0" as the destination bridge port or if there is no FDB entry
> at all, in which case the packet will be flooded.
>
> One of the attributes of the GRE device is the remote address, which is
> the encapsulating destination IP. Linux needs to have a route telling it
> how to reach this destination address or the packet will be dropped.
>
> > The other endpoint is on a different Linux system with another l2gre0
> > interface having IP address 10.10.10.1
> >
> > Thanks
> > N Gadre
> >

Re: Routing in case of GRE interface under a bridge

[Nayan Gadre]

messed up the diagram, here's another attempt.

System A
192.168.0.103 br1.1 <-- eth1
br0 | <---l2gre0
      | <---eth0  --> client connected (receives IP 10.10.10.105)

System B
192.168.0.10 eth0
10.10.10.1  l2gre0 <---dhcp server

On Thu, Jun 29, 2023 at 9:00 PM Nayan Gadre <beejoy.nayan@gmail.com> wrote:
>
> Yes, the l2gre0 on System A and System B is a gretap created using
> following command
>
> ip link add l2gre0 type gretap remote 192.168.0.10 local 192.168.0.103
>             --> and vice versa on System B.
>
> On system A, l2gre0 and eth0 are under a bridge br0. l2gre0 does not
> have an IP address.
> On system B, l2gre0 is independent but has IP address 10.10.10.1, and
> a DHCP server is running on it providing IP to clients connected
> through the tunnel.
>                       System A
>  |                               System B
>                                             192.168.0.103            |
>                      br0                         br1
>     |                        eth0
> l2gre0
>            eth0           l2gre0            eth1                     |
>                   192.168.0.10                        10.10.10.1
>
> On system A:
> / # ip r
> default via 192.168.0.10 dev br1.1
> 169.254.0.0/16 dev br1.1 proto kernel scope link src 169.254.32.107
> 192.168.0.0/24 dev br1.1 proto kernel scope link src 192.168.0.103
>
> On system B:
> ngadre@in01-7h4wrf3:~$ ip r
> default via 10.110.234.254 dev eno1 proto dhcp metric 100
> 10.10.10.0/24 dev l2gre0 proto kernel scope link src 10.10.10.1
> 192.168.0.0/24 dev enp3s0 proto kernel scope link src 192.168.0.10
>
> As we can see, on System B there is a route pointing at l2gre0.
> However, there is no such route on System A. Yet the packet gets
> encapsulated
> A client connected to eth0 on System A sends packet with destination
> 10.10.10.1 (def gateway). So I am guessing l2gre0 receives this packet
> since it gets flooded by br0 and even though System A not having a
> route to 10.10.10.0/24 it will encapsulate. Is this the behavior in
> case of a bridged tunnel interface ?.
>
> On Thu, Jun 29, 2023 at 4:42 PM Ido Schimmel <idosch@idosch.org> wrote:
> >
> > On Wed, Jun 28, 2023 at 07:06:45PM +0530, Nayan Gadre wrote:
> > > I have a "l2gre0" and "eth0" interface under the bridge "br0".
> >
> > I assume "l2gre0" is a gretap, not ipgre.
> >
> > > If a packet comes to eth0 interface with a destination IP address say
> > > 10.10.10.1 which is not known on the Linux system, as there is no
> > > route for 10.10.10.1, will the l2gre0 interface encapsulate this
> > > packet and send it across the tunnel ?
> >
> > The bridge doesn't care about IP addresses when forwarding unicast
> > packets. Forwarding happens based on DMAC. Packet will be transmitted
> > through "l2gre0" if the bridge has a matching FDB entry for the DMAC
> > with "l2gre0" as the destination bridge port or if there is no FDB entry
> > at all, in which case the packet will be flooded.
> >
> > One of the attributes of the GRE device is the remote address, which is
> > the encapsulating destination IP. Linux needs to have a route telling it
> > how to reach this destination address or the packet will be dropped.
> >
> > > The other endpoint is on a different Linux system with another l2gre0
> > > interface having IP address 10.10.10.1
> > >
> > > Thanks
> > > N Gadre
> > >

Re: Routing in case of GRE interface under a bridge

[Ido Schimmel]

(Please avoid top-posting).

On Thu, Jun 29, 2023 at 09:00:21PM +0530, Nayan Gadre wrote:
> Yes, the l2gre0 on System A and System B is a gretap created using
> following command
> 
> ip link add l2gre0 type gretap remote 192.168.0.10 local 192.168.0.103
>             --> and vice versa on System B.
> 
> On system A, l2gre0 and eth0 are under a bridge br0. l2gre0 does not
> have an IP address.

That's OK. It's meaningless to assign an IP address to a bridge port.

> On system B, l2gre0 is independent but has IP address 10.10.10.1, and
> a DHCP server is running on it providing IP to clients connected
> through the tunnel.
>                       System A
>  |                               System B
>                                             192.168.0.103            |
>                      br0                         br1
>     |                        eth0
> l2gre0
>            eth0           l2gre0            eth1                     |
>                   192.168.0.10                        10.10.10.1
> 
> On system A:
> / # ip r
> default via 192.168.0.10 dev br1.1
> 169.254.0.0/16 dev br1.1 proto kernel scope link src 169.254.32.107
> 192.168.0.0/24 dev br1.1 proto kernel scope link src 192.168.0.103
> 
> On system B:
> ngadre@in01-7h4wrf3:~$ ip r
> default via 10.110.234.254 dev eno1 proto dhcp metric 100
> 10.10.10.0/24 dev l2gre0 proto kernel scope link src 10.10.10.1
> 192.168.0.0/24 dev enp3s0 proto kernel scope link src 192.168.0.10
> 
> As we can see, on System B there is a route pointing at l2gre0.
> However, there is no such route on System A. Yet the packet gets
> encapsulated
> A client connected to eth0 on System A sends packet with destination
> 10.10.10.1 (def gateway). So I am guessing l2gre0 receives this packet
> since it gets flooded by br0 and even though System A not having a
> route to 10.10.10.0/24 it will encapsulate.

The overlay IP address is irrelevant. System A does not inspect it, it
simply forwards Ethernet frames (based on DMAC) between both bridge
ports - eth0 and l2gre0.

As to whether it gets flooded or not, it depends on the DMAC of the
frame received via eth0 and the FDB of br0. I expect the DMAC to be the
MAC of l2gre0 on system B. You can dump the FDB on system A using the
following command:

# bridge fdb show br br0 | grep master

> Is this the behavior in case of a bridged tunnel interface ?.

This is the encapsulation flow on system A as I understand it from your
data:

1. Ethernet packet is forwarded by br0 from eth0 to l2gre0.

2. l2gre encapsulates the Ethernet packet with
{sip=192.168.0.103,dip=192.168.0.10,ip_proto=0x2f,gre (proto=0x6558)}

3. Encapsulated packet is routed out of br.1 that has the most specific
route of 192.168.0.0/24 towards 192.168.0.10

Re: Routing in case of GRE interface under a bridge

[Nayan Gadre]

On Thu, Jun 29, 2023 at 10:29 PM Ido Schimmel <idosch@idosch.org> wrote:
>
> (Please avoid top-posting).
>
> On Thu, Jun 29, 2023 at 09:00:21PM +0530, Nayan Gadre wrote:
> > Yes, the l2gre0 on System A and System B is a gretap created using
> > following command
> >
> > ip link add l2gre0 type gretap remote 192.168.0.10 local 192.168.0.103
> >             --> and vice versa on System B.
> >
> > On system A, l2gre0 and eth0 are under a bridge br0. l2gre0 does not
> > have an IP address.
>
> That's OK. It's meaningless to assign an IP address to a bridge port.
>
> > On system B, l2gre0 is independent but has IP address 10.10.10.1, and
> > a DHCP server is running on it providing IP to clients connected
> > through the tunnel.
> >                       System A
> >  |                               System B
> >                                             192.168.0.103            |
> >                      br0                         br1
> >     |                        eth0
> > l2gre0
> >            eth0           l2gre0            eth1                     |
> >                   192.168.0.10                        10.10.10.1
> >
> > On system A:
> > / # ip r
> > default via 192.168.0.10 dev br1.1
> > 169.254.0.0/16 dev br1.1 proto kernel scope link src 169.254.32.107
> > 192.168.0.0/24 dev br1.1 proto kernel scope link src 192.168.0.103
> >
> > On system B:
> > ngadre@in01-7h4wrf3:~$ ip r
> > default via 10.110.234.254 dev eno1 proto dhcp metric 100
> > 10.10.10.0/24 dev l2gre0 proto kernel scope link src 10.10.10.1
> > 192.168.0.0/24 dev enp3s0 proto kernel scope link src 192.168.0.10
> >
> > As we can see, on System B there is a route pointing at l2gre0.
> > However, there is no such route on System A. Yet the packet gets
> > encapsulated
> > A client connected to eth0 on System A sends packet with destination
> > 10.10.10.1 (def gateway). So I am guessing l2gre0 receives this packet
> > since it gets flooded by br0 and even though System A not having a
> > route to 10.10.10.0/24 it will encapsulate.
>
> The overlay IP address is irrelevant. System A does not inspect it, it
> simply forwards Ethernet frames (based on DMAC) between both bridge
> ports - eth0 and l2gre0.
>
> As to whether it gets flooded or not, it depends on the DMAC of the
> frame received via eth0 and the FDB of br0. I expect the DMAC to be the
> MAC of l2gre0 on system B. You can dump the FDB on system A using the
> following command:
>
> # bridge fdb show br br0 | grep master
>
> > Is this the behavior in case of a bridged tunnel interface ?.
>
> This is the encapsulation flow on system A as I understand it from your
> data:
>
> 1. Ethernet packet is forwarded by br0 from eth0 to l2gre0.
>
> 2. l2gre encapsulates the Ethernet packet with
> {sip=192.168.0.103,dip=192.168.0.10,ip_proto=0x2f,gre (proto=0x6558)}
>
> 3. Encapsulated packet is routed out of br.1 that has the most specific
> route of 192.168.0.0/24 towards 192.168.0.10

Ok, so packet is forwarded from eth0 to l2gre0, and it would be internally
(by the kernel "br_flood/br_forward") pushed on the l2gre0 transmit
path via  "ipgre_xmit"