Re: Kernel Panic after updating Kernel

["dxiri@xirihosting.com" <dxiri@xirihosting.com>]

-----Original Message-----
From: "Jason A. Donenfeld" <Jason@zx2c4.com>
Sent: Wednesday, June 17, 2020 4:32am
To: "dxiri@xirihosting.com" <dxiri@xirihosting.com>
Cc: "WireGuard mailing list" <wireguard@lists.zx2c4.com>, "ElRepo" <contact@elrepo.org>
Subject: Re: Kernel Panic after updating Kernel

Hi Diego,

On Wed, Jun 17, 2020 at 2:01 AM dxiri@xirihosting.com
<dxiri@xirihosting.com> wrote:
>
> Posted this on IRC but got no response, probably this will be a better place:
>
> I updated my Centos7 server yesterday and wireguard is causing a kernel panic, wanted to know if this is a known issue?
>
> Using kernel 3.10.0-1127.10.1.el7.x86_64
>
> I Tried with 2 different repos (elrepo and Copr repo for wireguard owned by jdoss) and I have the same issue.
>
> I took a screenshot of The kernel panic and uploaded at https://imgur.com/a/Ojxeor0
>
> Another interesting bit of info is that as long as I don't move traffic trough wg0 vnic, no panic happens. I can easily trigger the panic by just doing a ping to the other VPN endpoint and I am able to reproduce this every single time.
>
> # lsmod | grep -i wire
> wireguard             200896  0
> ip6_udp_tunnel         12755  1 wireguard
> udp_tunnel             14423  1 wireguard
>
> Thanks for the help!
> Diego

Huh, that's funny -- I'm unable to reproduce the bug at all.

Does running this script crash for you?
https://salsa.debian.org/debian/wireguard-linux-compat/-/raw/debian/master/debian/tests/netns-mini

If not, could you describe your setup more and maybe some repro steps for me?

Thanks,
Jason

--------------------

Hi Jason, 

Tried your script, here is the result (spoiler...no crash):

root@box [4542 22:04:00 /etc/wireguard]# bash netns-mini-test.sh
[+] ip netns add wg-test-36633-0
[+] ip netns add wg-test-36633-1
[+] ip netns add wg-test-36633-2
[+] NS0: ip link set up dev lo
[+] NS0: ip link add dev wg0 type wireguard
[+] NS0: ip link set wg0 netns wg-test-36633-1
[+] NS0: ip link add dev wg0 type wireguard
[+] NS0: ip link set wg0 netns wg-test-36633-2
[+] NS1: ip addr add 192.168.241.1/24 dev wg0
[+] NS2: ip addr add 192.168.241.2/24 dev wg0
[+] wg genkey
[+] wg genkey
[+] wg pubkey
[+] wg pubkey
[+] NS1: wg set wg0 private-key /dev/fd/63 listen-port 1 peer 0zUBih0nFOHRDzl6mBxcHaCfwX+s9sE6rLgK4f8LdiU= allowed-ips 192.168.241.2/32
[+] NS2: wg set wg0 private-key /dev/fd/63 listen-port 2 peer jBLy+DQDc21/f5z9Yfz287FB/Z9PhaXvNdMw7WDoWFM= allowed-ips 192.168.241.1/32
[+] NS1: ip link set up dev wg0
[+] NS2: ip link set up dev wg0
[+] NS2: wg set wg0 peer jBLy+DQDc21/f5z9Yfz287FB/Z9PhaXvNdMw7WDoWFM= endpoint 127.0.0.1:1
[+] NS2: ping -c 10 -f -W 1 192.168.241.1
PING 192.168.241.1 (192.168.241.1) 56(84) bytes of data.

--- 192.168.241.1 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 1ms
rtt min/avg/max/mdev = 0.054/0.159/0.884/0.243 ms, ipg/ewma 0.209/0.316 ms
[+] NS0: ip link del dev wg0
[+] NS1: ip link del dev wg0
[+] NS2: ip link del dev wg0
[+] ip netns del wg-test-36633-1
[+] ip netns del wg-test-36633-2
[+] ip netns del wg-test-36633-0

About my setup:

1) KVM hosted VM
2) Using wg-quick, followed this tutorial: https://www.stavros.io/posts/how-to-configure-wireguard/
3) CPanel v88.0.10 (as far as I know, CPanel does NOT modify stock Centos 7 kernel)

4) root@box [4545 22:07:54 /etc/wireguard]# free -m
              total        used        free      shared  buff/cache   available
Mem:           2363        1373         174          12         815         793
Swap:          1999        1637         362

5) root@box [4547 22:10:37 /etc/wireguard]# cat wg0.conf
[Interface]
Address = 192.168.100.101/28
PrivateKey = 0000000xxxxxxxpjdlkkljkljalkjlkjl=
ListenPort = 11555

[Peer]
PublicKey = djkjadlkjlkjkldjlkjaslkjadlk=
AllowedIPs = 192.168.100.100/32
Endpoint = 1.1.1.1:11555

6) Yum operations trigger a lot of exclutions for elrepo, but nothing seems wireguard related:

Loaded plugins: changelog, elrepo, fastestmirror, priorities, tsflags, universal-hooks
Loading mirror speeds from cached hostfile
 * EA4: 208.100.0.204
 * cpanel-addons-production-feed: 208.100.0.204
 * cpanel-plugins: 208.100.0.204
 * elrepo: elrepo.0m3n.net
 * epel: mirror.csis.ysu.edu
[elrepo]: excluding package: kmod-3c59x-0.0-3.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-8188eu-4.1.4_6773.20130222-4.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-8188eu-4.1.4_6773.20130222-5.el7_6.elrepo.x86_64
[elrepo]: excluding package: kmod-8188eu-5.2.2.4-1.20190907git.el7_7.elrepo.x86_64
[elrepo]: excluding package: kmod-a2818-1.20-1.el7.elrepo.x86_64
[elrepo]: excluding package: kmod-a3818-1.6.0-1.el7.elrepo.x86_64
[elrepo]: excluding package: kmod-a3818-1.6.2-1.el7_6.elrepo.x86_64
[elrepo]: excluding package: kmod-aacraid-1.2.1-5.el7.elrepo.x86_64
[elrepo]: excluding package: kmod-aic7xxx-7.0-3.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-ar5523-0.0-8.el7_6.elrepo.x86_64
[elrepo]: excluding package: kmod-ar5523-0.0-9.el7_7.elrepo.x86_64
[elrepo]: excluding package: kmod-ath5k-0.0-12.el7_7.elrepo.x86_64
[elrepo]: excluding package: kmod-cassini-1.6-2.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-cciss-3.6.26-5.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-cciss-3.6.26-6.el7_6.elrepo.x86_64
[elrepo]: excluding package: kmod-cciss-3.6.26-7.el7_7.elrepo.x86_64
[elrepo]: excluding package: kmod-drbd84-8.4.11-1.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-drbd84-8.4.11-1.1.el7_6.elrepo.x86_64
[elrepo]: excluding package: kmod-drbd90-9.0.14-1.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-drbd90-9.0.16-1.el7_6.elrepo.x86_64
[elrepo]: excluding package: kmod-drbd90-9.0.20-1.el7_7.elrepo.x86_64
[elrepo]: excluding package: kmod-e100-3.5.24-3.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-ecryptfs-0.1-1.el7_6.elrepo.x86_64
[elrepo]: excluding package: kmod-forcedeth-0.64-3.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-fpga-mgr-0.0-1.el7_6.elrepo.x86_64
[elrepo]: excluding package: kmod-hfs-0.0-4.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-hfsplus-0.0-5.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-i2c-i801-0.0-4.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-i2c-i801-0.0-5.el7_6.elrepo.x86_64
[elrepo]: excluding package: kmod-i2c-i801-0.0-6.el7_6.elrepo.x86_64
[elrepo]: excluding package: kmod-ixgb-1.0.135-4.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-ixgbe-5.5.5-1.el7_6.elrepo.x86_64
[elrepo]: excluding package: kmod-ixgbe-5.6.3-1.el7_7.elrepo.x86_64
[elrepo]: excluding package: kmod-ixgbe-5.6.3-2.el7_7.elrepo.x86_64
[elrepo]: excluding package: kmod-joydev-0.0-4.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-mt7601u-4.14.108-1.el7_6.elrepo.x86_64
[elrepo]: excluding package: kmod-mt7601u-4.14.108-2.el7_7.elrepo.x86_64
[elrepo]: excluding package: kmod-nct6775-0.0-4.20180327git.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-nct6775-0.0-5.el7_7.elrepo.x86_64
[elrepo]: excluding package: kmod-ne2k-pci-1.03-4.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-netatop-0.3-4.el7_6.elrepo.x86_64
[elrepo]: excluding package: kmod-netatop-2.0-1.el7_6.elrepo.x86_64
[elrepo]: excluding package: kmod-niu-1.1-2.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-nvidia-440.44-1.el7_7.elrepo.x86_64
[elrepo]: excluding package: nvidia-x11-drv-libs-440.44-1.el7_7.elrepo.x86_64
[elrepo]: excluding package: nvidia-x11-drv-libs-440.44-1.el7_7.elrepo.i686
[elrepo]: excluding package: nvidia-x11-drv-440.44-1.el7_7.elrepo.x86_64
[elrepo]: excluding package: kmod-nvidia-440.59-1.el7_7.elrepo.x86_64
[elrepo]: excluding package: nvidia-x11-drv-libs-440.59-1.el7_7.elrepo.x86_64
[elrepo]: excluding package: nvidia-x11-drv-440.59-1.el7_7.elrepo.x86_64
[elrepo]: excluding package: nvidia-x11-drv-libs-440.59-1.el7_7.elrepo.i686
[elrepo]: excluding package: kmod-nvidia-440.64-1.el7_7.elrepo.x86_64
[elrepo]: excluding package: nvidia-x11-drv-libs-440.64-1.el7_7.elrepo.x86_64
[elrepo]: excluding package: nvidia-x11-drv-libs-440.64-1.el7_7.elrepo.i686
[elrepo]: excluding package: nvidia-x11-drv-440.64-1.el7_7.elrepo.x86_64
[elrepo]: excluding package: kmod-nvidia-340xx-340.107-2.el7_6.elrepo.x86_64
[elrepo]: excluding package: kmod-nvidia-340xx-340.107-3.el7_7.elrepo.x86_64
[elrepo]: excluding package: kmod-nvidia-390xx-390.116-1.el7_6.elrepo.x86_64
[elrepo]: excluding package: nvidia-x11-drv-390xx-390.116-1.el7_6.elrepo.x86_64
[elrepo]: excluding package: nvidia-x11-drv-390xx-libs-390.116-1.el7_6.elrepo.x86_64
[elrepo]: excluding package: nvidia-x11-drv-390xx-libs-390.116-1.el7_6.elrepo.i686
[elrepo]: excluding package: kmod-nvidia-390xx-390.116-2.el7_7.elrepo.x86_64
[elrepo]: excluding package: nvidia-x11-drv-390xx-libs-390.116-2.el7_7.elrepo.i686
[elrepo]: excluding package: nvidia-x11-drv-390xx-390.116-2.el7_7.elrepo.x86_64
[elrepo]: excluding package: nvidia-x11-drv-390xx-libs-390.116-2.el7_7.elrepo.x86_64
[elrepo]: excluding package: kmod-r8168-8.046.00-1.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-r8168-8.047.04-1.el7_7.elrepo.x86_64
[elrepo]: excluding package: kmod-r8168-8.047.05-1.el7_7.elrepo.x86_64
[elrepo]: excluding package: kmod-r8168-8.048.00-1.el7_7.elrepo.x86_64
[elrepo]: excluding package: kmod-r8169-6.020.00-3.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-r8822be-0.0-1.el7_6.elrepo.x86_64
[elrepo]: excluding package: kmod-reiserfs-0.0-3.el7_6.elrepo.x86_64
[elrepo]: excluding package: kmod-rr62x-1.2-3.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-rr64xl-1.4.0-1.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-rtl8812au-5.3.4-1.el7_7.elrepo.x86_64
[elrepo]: excluding package: kmod-sis190-1.4-1.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-sis900-1.08.10-2.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-sym53c8xx-0.0-4.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-tpe-2.0.3-6.20170731git.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-tpe-2.0.4-1.el7_7.elrepo.x86_64
[elrepo]: excluding package: kmod-typhoon-1.0-3.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-usbip-1.0.1-2.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-usbip-1.0.1-3.el7_6.elrepo.x86_64
[elrepo]: excluding package: kmod-usbip-1.0.1-4.el7_7.elrepo.x86_64
[elrepo]: excluding package: kmod-via-rhine-1.5.1-3.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-via-velocity-1.15-2.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-xpad-0.0.6-7.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-xpad-0.0.6-8.el7_6.elrepo.x86_64
[elrepo]: excluding package: kmod-xpad-0.0.6-9.el7_7.elrepo.x86_64
[elrepo]: excluding package: kmod-zd1211rw-1.0-6.el7_5.elrepo.x86_64
[elrepo]: excluding package: kmod-zd1211rw-1.0-7.el7_7.elrepo.x86_64
178 packages excluded due to repository priority protections

7) I have another VM with another service provider and have the exact same issue after updating. This other VM has a free version of CPanel called DNSONLY, if you care to install to take a shot at reproducing: https://docs.cpanel.net/installation-guide/cpanel-dnsonly-installation/

Any other info you need feel free to ask :)

Thanks!
Diego