Unable to connect to eduroam

Unable to connect to eduroam

[Michael Yartys]

Hi

I'm running iwd 2.14 on Fedora 39 Silverblue with kernel version 6.7.5-200.fc39.x86_64, and I've been having issues for the last two weeks or so connecting to my university eduroam network. What happens is that I get an internal error that causes the connection attempt to fail with eapFail. Here's the log output with TLS debugging enabled:

Feb 23 14:35:55 localhost iwd[1233]: TTLS: tls_tx_handshake:1244 Sending a TLS_CLIENT_HELLO of 140 bytes
Feb 23 14:35:55 localhost iwd[1233]: TTLS: l_tls_start:3610 New state TLS_HANDSHAKE_WAIT_HELLO
Feb 23 14:35:55 localhost iwd[1233]: TTLS: tls_handle_handshake:3074 Handling a TLS_SERVER_HELLO of 45 bytes
Feb 23 14:35:55 localhost iwd[1233]: TTLS: tls_handle_server_hello:2419 Negotiated TLS 1.2
Feb 23 14:35:55 localhost iwd[1233]: TTLS: tls_handle_server_hello:2455 Negotiated TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
Feb 23 14:35:55 localhost iwd[1233]: TTLS: tls_handle_server_hello:2466 Negotiated CompressionMethod.null
Feb 23 14:35:55 localhost iwd[1233]: TTLS: tls_handle_server_hello:2492 New state TLS_HANDSHAKE_WAIT_CERTIFICATE
Feb 23 14:35:55 localhost iwd[1233]: TTLS: tls_handle_handshake:3074 Handling a TLS_CERTIFICATE of 2126 bytes
Feb 23 14:35:55 localhost iwd[1233]: TTLS: tls_handle_certificate:2562 Peer certchain written to /tmp/iwd-tls-debug-server-cert.pem
Feb 23 14:35:55 localhost iwd[1233]: TTLS: tls_handle_certificate:2673 Disconnect desc=internal_error local-desc=close_notify reason=Can't l_key_get_info for peer public key
Feb 23 14:35:55 localhost iwd[1233]: TTLS: tls_send_alert:1175 Sending a Fatal Alert: internal_error
Feb 23 14:35:55 localhost iwd[1233]: TTLS: tls_reset_handshake:195 New state TLS_HANDSHAKE_WAIT_START
Feb 23 14:35:55 localhost iwd[1233]: TTLS: Tunnel has disconnected with alert: internal_error
Feb 23 14:35:56 localhost iwd[1233]: EAP completed with eapFail
Feb 23 14:35:56 localhost iwd[1233]: TTLS: tls_reset_handshake:195 New state TLS_HANDSHAKE_WAIT_START
Feb 23 14:35:56 localhost iwd[1233]: TTLS: tls_reset_handshake:195 New state TLS_HANDSHAKE_WAIT_START
Feb 23 14:35:56 localhost iwd[1233]: 4-Way handshake failed for ifindex: 4, reason: 23


Here is the associated dumped peer certchain:

-----BEGIN CERTIFICATE-----
MIIExDCCBGugAwIBAgIRAM+MIw8HuugW44b38b8768QwCgYIKoZIzj0EAwIwRDEL
MAkGA1UEBhMCTkwxGTAXBgNVBAoTEEdFQU5UIFZlcmVuaWdpbmcxGjAYBgNVBAMT
EUdFQU5UIE9WIEVDQyBDQSA0MB4XDTI0MDIwNjAwMDAwMFoXDTI1MDIwNTIzNTk1
OVowWTELMAkGA1UEBhMCTk8xDTALBgNVBAgTBE9zbG8xHTAbBgNVBAoTFFVuaXZl
cnNpdGV0ZXQgaSBPc2xvMRwwGgYDVQQDExNyYWRpdXMtZWR1MDEudWlvLm5vMFkw
EwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAExF7Q2VsO6h0fciizxw+/4ptSYFo6gnUy
8hHvw766pf2B/fWKgG8BBXR2refZHibmSh3kF3tibYH4t4HULJp4xqOCAycwggMj
MB8GA1UdIwQYMBaAFO20oDNqGwiRtr36QZK9mqurY/RTMB0GA1UdDgQWBBTSHfh9
sY1lBcBLdLbTT+9zurmgLTAOBgNVHQ8BAf8EBAMCB4AwDAYDVR0TAQH/BAIwADAd
BgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwSQYDVR0gBEIwQDA0BgsrBgEE
AbIxAQICTzAlMCMGCCsGAQUFBwIBFhdodHRwczovL3NlY3RpZ28uY29tL0NQUzAI
BgZngQwBAgIwPwYDVR0fBDgwNjA0oDKgMIYuaHR0cDovL0dFQU5ULmNybC5zZWN0
aWdvLmNvbS9HRUFOVE9WRUNDQ0E0LmNybDB1BggrBgEFBQcBAQRpMGcwOgYIKwYB
BQUHMAKGLmh0dHA6Ly9HRUFOVC5jcnQuc2VjdGlnby5jb20vR0VBTlRPVkVDQ0NB
NC5jcnQwKQYIKwYBBQUHMAGGHWh0dHA6Ly9HRUFOVC5vY3NwLnNlY3RpZ28uY29t
MIIBfwYKKwYBBAHWeQIEAgSCAW8EggFrAWkAdwDPEVbu1S58r/OHW9lpLpvpGnFn
SrAX7KwB0lt3zsw7CAAAAY191L4wAAAEAwBIMEYCIQC6zqKLAVShWFlASZuktoGx
eddmBUZ5CR5gQxpjo01AIwIhANDuD6ZGlQeMi1wPMMrT+6nLL+JSb41/C2e+0CEj
ZUU/AHUAouMK5EXvva2bfjjtR2d3U9eCW4SU1yteGyzEuVCkR+cAAAGNfdS+8gAA
BAMARjBEAiAbl1nI5yax0AtxqxGrUud42ZdtNihV1mJZsoFESkGMJwIgSldsoiAw
qh5bZOCwGU74cOGA4MUmHtOzYsmH1GsK8g4AdwBOdaMnXJoQwzhbbNTfP1LrHfDg
jhuNacCx+mSxYpo53wAAAY191L5aAAAEAwBIMEYCIQDe23Hd8H0j3rpBcAMuOrj1
HO3DV8/irDdOtwsNWCFawAIhAK8Xrlq12GIfmyGfAATLb2qKfiAYpjxjU32zkrlS
zAWPMB4GA1UdEQQXMBWCE3JhZGl1cy1lZHUwMS51aW8ubm8wCgYIKoZIzj0EAwID
RwAwRAIgGbiOSzrozjf9HDKDQKcGPPVSlMz8CPE72GDkNn2adygCIGozGfF38Yqb
f01HpBCMJ2D/3F54kLeWevwEiecMULn1
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIDeTCCAv+gAwIBAgIRAOuOgRlxKfSvZO+BSi9QzukwCgYIKoZIzj0EAwMwgYgx
CzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpOZXcgSmVyc2V5MRQwEgYDVQQHEwtKZXJz
ZXkgQ2l0eTEeMBwGA1UEChMVVGhlIFVTRVJUUlVTVCBOZXR3b3JrMS4wLAYDVQQD
EyVVU0VSVHJ1c3QgRUNDIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTIwMDIx
ODAwMDAwMFoXDTMzMDUwMTIzNTk1OVowRDELMAkGA1UEBhMCTkwxGTAXBgNVBAoT
EEdFQU5UIFZlcmVuaWdpbmcxGjAYBgNVBAMTEUdFQU5UIE9WIEVDQyBDQSA0MFkw
EwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEXYkvGrfrMs2IwdI5+IwpEwPh+igW/BOW
etmOwP/ZIXC8fNeC3/ZYPAAMyRpFS0v3/c55FDTE2xbOUZ5zeVZYQqOCAYswggGH
MB8GA1UdIwQYMBaAFDrhCYbUzxnClnZ0SXbc4DXGY2OaMB0GA1UdDgQWBBTttKAz
ahsIkba9+kGSvZqrq2P0UzAOBgNVHQ8BAf8EBAMCAYYwEgYDVR0TAQH/BAgwBgEB
/wIBADAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwOAYDVR0gBDEwLzAt
BgRVHSAAMCUwIwYIKwYBBQUHAgEWF2h0dHBzOi8vc2VjdGlnby5jb20vQ1BTMFAG
A1UdHwRJMEcwRaBDoEGGP2h0dHA6Ly9jcmwudXNlcnRydXN0LmNvbS9VU0VSVHJ1
c3RFQ0NDZXJ0aWZpY2F0aW9uQXV0aG9yaXR5LmNybDB2BggrBgEFBQcBAQRqMGgw
PwYIKwYBBQUHMAKGM2h0dHA6Ly9jcnQudXNlcnRydXN0LmNvbS9VU0VSVHJ1c3RF
Q0NBZGRUcnVzdENBLmNydDAlBggrBgEFBQcwAYYZaHR0cDovL29jc3AudXNlcnRy
dXN0LmNvbTAKBggqhkjOPQQDAwNoADBlAjAfs9nsM0qaJGVu6DpWVy4qojiOpwV1
h/MWZ5GJxy6CKv3+RMB3STkaFh0+Hifbk24CMQDRf/ujXAQ1b4nFpZGaSIKldygc
dCDAxbAd9tlxcN/+J534CJDblzd/40REzGWwS5k=
-----END CERTIFICATE-----


I'm sure these certificates are correct because I was able to roll back to a working build on Fedora Silverblue to keep connecting to the network. Unfortunately, I was dumb enough to forget to pin/freeze that working build to prevent it from being replaced by new versions as I continually updated the system.

I searched a bit around and found that some other folks had experienced the same issue on Arch: https://bbs.archlinux.org/viewtopic.php?id=291921

That thread links to another thread about the deprecation of SHA-1 in the kernel: https://bbs.archlinux.org/viewtopic.php?id=292208

rochus from the first thread was able to connect to their eduroam network when they tried a kernel with the offending commit reverted. Now, I know very little about certificates, but no certificates in my cert chain are SHA-1 signed, are they? Could this still be related to the issue I'm experiencing?


Michael

Re: Unable to connect to eduroam

[James Prestwood]

HI Michael,

On 2/23/24 6:25 AM, Michael Yartys wrote:
> Hi
>
> I'm running iwd 2.14 on Fedora 39 Silverblue with kernel version 6.7.5-200.fc39.x86_64, and I've been having issues for the last two weeks or so connecting to my university eduroam network. What happens is that I get an internal error that causes the connection attempt to fail with eapFail. Here's the log output with TLS debugging enabled:
<snip>
>
>
> I'm sure these certificates are correct because I was able to roll back to a working build on Fedora Silverblue to keep connecting to the network. Unfortunately, I was dumb enough to forget to pin/freeze that working build to prevent it from being replaced by new versions as I continually updated the system.
>
> I searched a bit around and found that some other folks had experienced the same issue on Arch: https://bbs.archlinux.org/viewtopic.php?id=291921
>
> That thread links to another thread about the deprecation of SHA-1 in the kernel: https://bbs.archlinux.org/viewtopic.php?id=292208
>
> rochus from the first thread was able to connect to their eduroam network when they tried a kernel with the offending commit reverted. Now, I know very little about certificates, but no certificates in my cert chain are SHA-1 signed, are they? Could this still be related to the issue I'm experiencing?

Yep it appears the issue is a lack of SHA1 in the kernel.

I'm not that familiar with TLS but based on the code RSA certs use 
PKCS1, which requires SHA1 (someone correct me if I'm wrong).

The kernel apparently removed this... they did (or are doing) the same 
thing for MD5. I guess we need to roll our own.

Thanks,

James

>
>
> Michael
>

Re: Unable to connect to eduroam

[Denis Kenzior]

Hi Michael,

> 
> I'm sure these certificates are correct because I was able to roll back to a working build on Fedora Silverblue to keep connecting to the network. Unfortunately, I was dumb enough to forget to pin/freeze that working build to prevent it from being replaced by new versions as I continually updated the system.

Do you have any clue what changes you made to the working build?  iwd hasn't 
touched certificate bits for a number of releases now.

> 
> I searched a bit around and found that some other folks had experienced the same issue on Arch: https://bbs.archlinux.org/viewtopic.php?id=291921
> 
> That thread links to another thread about the deprecation of SHA-1 in the kernel: https://bbs.archlinux.org/viewtopic.php?id=292208

Someone should complain to the linux-crypto folks for breaking uapis and have 
that change reverted.

> 
> rochus from the first thread was able to connect to their eduroam network when they tried a kernel with the offending commit reverted. Now, I know very little about certificates, but no certificates in my cert chain are SHA-1 signed, are they? Could this still be related to the issue I'm experiencing?

I can't dump the second certificate for some reason, but the first has:
[denkenz@archdev ell]$ openssl x509 -in cert2.pem -text
...
     Signature Algorithm: ecdsa-with-SHA256

Do you have CONFIG_ECDSA=y in your kernel?

[denkenz@archdev ell]$ zless /proc/config.gz | grep ECDSA
CONFIG_CRYPTO_ECDSA=y
CONFIG_MODULE_SIG_KEY_TYPE_ECDSA=y

Regards,
-Denis

Re: Unable to connect to eduroam

[Michael Yartys]

Hi Denis,
 
> 
> Hi Michael,
> 
> > I'm sure these certificates are correct because I was able to roll back to a working build on Fedora Silverblue to keep connecting to the network. Unfortunately, I was dumb enough to forget to pin/freeze that working build to prevent it from being replaced by new versions as I continually updated the system.
> 
> 
> Do you have any clue what changes you made to the working build? iwd hasn't
> touched certificate bits for a number of releases now.

Just to be clear, I didn't build anything myself, I just updated my system as you would on a day to day basis. My guess is that a new kernel version came along that messed something up.

> 
> > I searched a bit around and found that some other folks had experienced the same issue on Arch: https://bbs.archlinux.org/viewtopic.php?id=291921
> > 
> > That thread links to another thread about the deprecation of SHA-1 in the kernel: https://bbs.archlinux.org/viewtopic.php?id=292208
> 
> 
> Someone should complain to the linux-crypto folks for breaking uapis and have
> that change reverted.
> 
> > rochus from the first thread was able to connect to their eduroam network when they tried a kernel with the offending commit reverted. Now, I know very little about certificates, but no certificates in my cert chain are SHA-1 signed, are they? Could this still be related to the issue I'm experiencing?
> 
> 
> I can't dump the second certificate for some reason, but the first has:
> [denkenz@archdev ell]$ openssl x509 -in cert2.pem -text
> ...
> Signature Algorithm: ecdsa-with-SHA256
> 
> Do you have CONFIG_ECDSA=y in your kernel?
> 
> [denkenz@archdev ell]$ zless /proc/config.gz | grep ECDSA
> CONFIG_CRYPTO_ECDSA=y
> CONFIG_MODULE_SIG_KEY_TYPE_ECDSA=y

This is my config. As you can see, the first variable is set to yes, but the second one is not set.

michael@localhost:~$ cat /usr/lib/modules/6.7.5-200.fc39.x86_64/config | grep ECDSA
CONFIG_CRYPTO_ECDSA=y
# CONFIG_MODULE_SIG_KEY_TYPE_ECDSA is not set

> 
> Regards,
> -Denis

Michael

Re: Unable to connect to eduroam

[Denis Kenzior]

Hi Michael,
> 
> This is my config. As you can see, the first variable is set to yes, but the second one is not set.

Okay, CONFIG_CRYPTO_ECDSA should be enough.  I can't dump out one of the two 
certificates you have.  Do you have the CA certificate?  If so you can try running

	ell.git/tools/certchain-verify <ca file> <certchain file>

and see what happens.  If your only change was a kernel upgrade, then I would 
suspect sha1 removal as the culprit.  Boot to an earlier kernel and see?

If it does turn out to be sha1 deprecation, then this is a bug report for your 
distro and upstream kernel folks.  They need to stop breaking uapis.  SHA1 
signatures, while no longer secure, are still widespread.

Regards,
-Denis

Re: Unable to connect to eduroam

[Michael Yartys]

Hi Denis

> 
> > This is my config. As you can see, the first variable is set to yes, but the second one is not set.
> 
> 
> Okay, CONFIG_CRYPTO_ECDSA should be enough. I can't dump out one of the two
> certificates you have. Do you have the CA certificate? If so you can try running
> 
> ell.git/tools/certchain-verify <ca file> <certchain file>
> 
> 
> and see what happens. If your only change was a kernel upgrade, then I would
> suspect sha1 removal as the culprit. Boot to an earlier kernel and see?

As far as I understand, this would be the CA certificate, right? https://eduroam.no/connect/download/?idp=297;profile=430;os=x-pem

I tried to compile ell, but I've run into some issues. Do you have a compiled certchain-verify on hand? I could give it a try if you send it to me (by link or whatever works).

I took a look at the kernel changelog and found out that version 6.7 removed sha1 support (just search for "crypto: pkcs7 - remove sha1 support"): https://cdn.kernel.org/pub/linux/kernel/v6.x/ChangeLog-6.7

I'll try to downgrade to version 6.6.14 and give it a try on Monday when I'm back at the uni.

> 
> If it does turn out to be sha1 deprecation, then this is a bug report for your
> distro and upstream kernel folks. They need to stop breaking uapis. SHA1
> signatures, while no longer secure, are still widespread.
> 
> Regards,
> -Denis

Michael

Re: Unable to connect to eduroam

[Michael Yartys]

Hi

> I'll try to downgrade to version 6.6.14 and give it a try on Monday when I'm back at the uni.

Just wanted to say that it works with kernel version 6.6.14. Guess I'll have to report this upstream.

Michael