Re: Outgoing ping required in container environment (even with PersistentKeepalive)

[Nico Schottelius <nico.schottelius@ungleich.ch>]

A follow up: we "upgraded" the wireguard container to use the following
entrypoint.sh instead:

--------------------------------------------------------------------------------
set -x

# Ensure everything is clean / show prior state
wg show

# Start all definitions
for conf in /etc/wireguard/*.conf; do
    # Remove leftovers??
    wg-quick down $conf

    # Try to up and if any tunnel fails -> exit
    wg-quick up "$conf" || (sleep 300; exit 1)
done

# Debug output
while true; do
    # Establish tunnels, keepalive alone is not enough for some reason
    ping -c2 185.203.112.1
    ping -c2 2a0a:e5c0::
    wg show
    sleep 300
done
--------------------------------------------------------------------------------

This establishes the connection reliably. I guess that should not be the
case, but effectively, it is.

I was for a moment suspecting that the old container is overlapping
running while the new one is created and whether there is a race
condition, however two points speak against that being the source of the
problem:

- Using the "Recreate strategy" in k8s, the container is first shut
  down. Using this, the behaviour does not change
- Even if my assumption was right, I'd expect a new handshake at some
  point to happen, but even minutes after restarting the container, the
  IPv4 address is not reachable.

Best regards,

Nico

Nico Schottelius <nico.schottelius@ungleich.ch> writes:

> Good morning,
>
> another day news from the container land. When running wireguard in
> kubernetes, deleting the existing pod and replacing it with a new one, I
> see the following behaviour:
>
> - The assigned IPv4 address stops being reachable (good so far)
> - The assigned IPv4 address is then shortly reachable for about 5 seconds
> - The assigned IPv4 address stops being reachable (not good)
> - The assigned IPv4 address is again reachable, if I trigger a ping
>   through the tunnel inside the container (ok, but why?)
>
> I am using the following configuration:
>
> --------------------------------------------------------------------------------
> [Interface]
> PrivateKey = ...
> ListenPort = 51828
> Address = 185.155.29.81/32
> PostUp = iptables -t nat -I POSTROUTING -o ipv4 -j MASQUERADE
>
> # upstream
> [Peer]
> Endpoint = vpn-...:51820
> PublicKey = 6BRnQ+dmeFzVCH9RbM1pbJ7u3y3qrl+zUzzYCmC88kE=
> AllowedIPs = 0.0.0.0/1, 128.0.0.0/1
> PersistentKeepalive = 25
> --------------------------------------------------------------------------------
>
> And the following container specification:
>
> --------------------------------------------------------------------------------
>   spec:
>       containers:
>         - name: wireguard
>           image: ungleich/ungleich-wireguard:{{ $.Chart.AppVersion }}
>           # We only support 1 listener at the moment
>           # Outgoing connections are not affected
>           ports:
>             - containerPort: 51820
>           securityContext:
>             capabilities:
>               # NET_ADMIN for wg
>               # NET_RAW for iptables
>               add: ["NET_ADMIN", "NET_RAW" ]
>           volumeMounts:
>             - name: wireguard-config
>               mountPath: "/etc/wireguard"
>           resources:
>             requests:
>               memory: {{ $v.memory | default "1Gi" }}
>               cpu: {{ $v.cpu | default "1000m" }}
>             limits:
>               memory: {{ $v.memory | default "1Gi" }}
>               cpu: {{ $v.cpu | default "1000m" }}
> --------------------------------------------------------------------------------
>
> The strange thing is that after issuing the ping once inside the
> container:
>
> --------------------------------------------------------------------------------
> [8:41] nb2:~% kubectl -n wireguard exec -ti wireguard-vpn-server-7db664db6f-zl4fz -- ping -c2 -4 google.com
> PING google.com (172.217.168.78): 56 data bytes
> 64 bytes from 172.217.168.78: seq=0 ttl=116 time=9.110 ms
> 64 bytes from 172.217.168.78: seq=1 ttl=116 time=6.664 ms
>
> --- google.com ping statistics ---
> 2 packets transmitted, 2 packets received, 0% packet loss
> round-trip min/avg/max = 6.664/7.887/9.110 ms
> --------------------------------------------------------------------------------
>
> The connection stays correctly established.
>
> If anyone has a pointer on what might be going on, any help is
> appreciated.
>
> Best regards,
>
> Nico


--
Sustainable and modern Infrastructures by ungleich.ch