wifi: mac80211: unencrypted message in 4-way handshake

wifi: mac80211: unencrypted message in 4-way handshake

[Sperling, Tobias]

Hello,

recently we moved from a 4.9 kernel to 5.4 kernel and observed the following issue. To rule out that the issue was already fixed we then tried it with a 6.6 kernel and observed the same issue there, too. To sum it up, the following issue can be observed with kernel 5.4 and 6.6 but not with 4.9.

We are using a Cisco AP with EWC (C9210) and EAP-PEAP as authentication method. Additionally, a session timeout is configured in the AP (300 s) after which a rekeying is taking place. So every 5 minutes a new four-way handshake is taking place which usually is completely encrypted. However, while sniffing with an external device we saw that the fourth packet of the four-way handshake is send unencrypted by our clients, when it comes to EAP-PEAP. The first three packets are encrypted as expected. In case of the Cisco AP the attempt is then rejected with a 4WAY_HANDSHAKE_TIMEOUT leading to a completely new authentication.

On our clients we are using wpa_supplicant 2.10 with the same config on all kernels and therefore would rule out an issue in the supplicant, as it is working with the same supplicant in kernel 4.9.
As driver we are using ath9k for QCA2066 (starting at kernel 4.9) and rtw88 for RTL8822CE (backported, starting at kernel 5.4). The issue exists for both drivers for kernel 5.4 and 6.6. Furthermore, we were also able to test a proprietary driver for the RTL8822CE chipset on kernel 5.4 where the issue does not appear. However, latter driver is a full-mac driver not using the mac80211 driver in comparison to the soft-mac drivers of the mainline. Considering all the details we highly assume that the issue is to be found in the mac80211 driver.

Do you have some hints where to look at to solve the issue? Otherwise please let us know if we can provide some logs or config files.

Thanks in advance

Tobias Sperling

Re: wifi: mac80211: unencrypted message in 4-way handshake

[Kalle Valo]

"Sperling, Tobias" <Tobias.Sperling@Softing.com> writes:

> Hello,
>
> recently we moved from a 4.9 kernel to 5.4 kernel and observed the
> following issue. To rule out that the issue was already fixed we then
> tried it with a 6.6 kernel and observed the same issue there, too. To
> sum it up, the following issue can be observed with kernel 5.4 and 6.6
> but not with 4.9.
>
> We are using a Cisco AP with EWC (C9210) and EAP-PEAP as
> authentication method. Additionally, a session timeout is configured
> in the AP (300 s) after which a rekeying is taking place. So every 5
> minutes a new four-way handshake is taking place which usually is
> completely encrypted. However, while sniffing with an external device
> we saw that the fourth packet of the four-way handshake is send
> unencrypted by our clients, when it comes to EAP-PEAP. The first three
> packets are encrypted as expected. In case of the Cisco AP the attempt
> is then rejected with a 4WAY_HANDSHAKE_TIMEOUT leading to a completely
> new authentication.
>
> On our clients we are using wpa_supplicant 2.10 with the same config
> on all kernels and therefore would rule out an issue in the
> supplicant, as it is working with the same supplicant in kernel 4.9.
> As driver we are using ath9k for QCA2066 (starting at kernel 4.9) and

QCA2066 with ath9k? ath9k is a driver for 802.11n hardware and QCA2066
is 802.11ax generation, how would that work?

-- 
https://patchwork.kernel.org/project/linux-wireless/list/

https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches

Re: wifi: mac80211: unencrypted message in 4-way handshake

[Sperling, Tobias]

> > Hello,
> >
> > recently we moved from a 4.9 kernel to 5.4 kernel and observed the
> > following issue. To rule out that the issue was already fixed we then
> > tried it with a 6.6 kernel and observed the same issue there, too. To
> > sum it up, the following issue can be observed with kernel 5.4 and 6.6
> > but not with 4.9.
> >
> > We are using a Cisco AP with EWC (C9210) and EAP-PEAP as
> > authentication method. Additionally, a session timeout is configured
> > in the AP (300 s) after which a rekeying is taking place. So every 5
> > minutes a new four-way handshake is taking place which usually is
> > completely encrypted. However, while sniffing with an external device
> > we saw that the fourth packet of the four-way handshake is send
> > unencrypted by our clients, when it comes to EAP-PEAP. The first three
> > packets are encrypted as expected. In case of the Cisco AP the attempt
> > is then rejected with a 4WAY_HANDSHAKE_TIMEOUT leading to a
> completely
> > new authentication.
> >
> > On our clients we are using wpa_supplicant 2.10 with the same config
> > on all kernels and therefore would rule out an issue in the
> > supplicant, as it is working with the same supplicant in kernel 4.9.
> > As driver we are using ath9k for QCA2066 (starting at kernel 4.9) and
> 
> QCA2066 with ath9k? ath9k is a driver for 802.11n hardware and QCA2066
> is 802.11ax generation, how would that work?

Right, somehow I collected the wrong info for the ath9k driven device. Sorry
about that. It's an AR958x adapter for the ath9k driver.

Re: wifi: mac80211: unencrypted message in 4-way handshake

[Johannes Berg]

On Mon, 2024-06-10 at 13:50 +0000, Sperling, Tobias wrote:
> 
> recently we moved from a 4.9 kernel to 5.4 kernel

🤯️

> Do you have some hints where to look at to solve the issue?

No, way too old.

Take a look at supplicant logs in how it behaves differently, I guess.

The only thing that comes to mind is that *maybe* the eapol-over-nl80211
path gets used (different kernels) and that has different behaviour in
wpa_s, but honestly, no idea.

johannes

Re: wifi: mac80211: unencrypted message in 4-way handshake

[Sperling, Tobias]

> > recently we moved from a 4.9 kernel to 5.4 kernel
> 
> 🤯️

:D
> 
> > Do you have some hints where to look at to solve the issue?
> 
> No, way too old.

That's why we also tried it with kernel 6.6, where we faced the same issue. So I'd
appreciate hints for kernel 6.6 (or probably later), too. :)
> 
> Take a look at supplicant logs in how it behaves differently, I guess.
> 
> The only thing that comes to mind is that *maybe* the eapol-over-nl80211
> path gets used (different kernels) and that has different behaviour in
> wpa_s, but honestly, no idea.

Thanks, will have a look there.
>
> johannes

Re: wifi: mac80211: unencrypted message in 4-way handshake

[Johannes Berg]

On Tue, 2024-06-11 at 12:36 +0000, Sperling, Tobias wrote:
> > > recently we moved from a 4.9 kernel to 5.4 kernel
> > 
> > 🤯️
> 
> :D
> > 
> > > Do you have some hints where to look at to solve the issue?
> > 
> > No, way too old.
> 
> That's why we also tried it with kernel 6.6, where we faced the same issue. So I'd
> appreciate hints for kernel 6.6 (or probably later), too. :)

Yeah but even moving 4.9->6.6 is a huge range, even bigger :)

> > Take a look at supplicant logs in how it behaves differently, I guess.
> > 
> > The only thing that comes to mind is that *maybe* the eapol-over-nl80211
> > path gets used (different kernels) and that has different behaviour in
> > wpa_s, but honestly, no idea.
> 
> Thanks, will have a look there.

I would certainly hope that wpa_s uses that path on the new kernel. It
wasn't in 4.9, pretty sure.

johannes
> 

RE: wifi: mac80211: unencrypted message in 4-way handshake

[Sperling, Tobias]

> > > > recently we moved from a 4.9 kernel to 5.4 kernel
> > >
> > > 🤯️
> >
> > :D
> > >
> > > > Do you have some hints where to look at to solve the issue?
> > >
> > > No, way too old.
> >
> > That's why we also tried it with kernel 6.6, where we faced the same issue. So
> I'd
> > appreciate hints for kernel 6.6 (or probably later), too. :)
> 
> Yeah but even moving 4.9->6.6 is a huge range, even bigger :)
> 
> > > Take a look at supplicant logs in how it behaves differently, I guess.
> > >
> > > The only thing that comes to mind is that *maybe* the eapol-over-nl80211
> > > path gets used (different kernels) and that has different behaviour in
> > > wpa_s, but honestly, no idea.
> >
> > Thanks, will have a look there.
> 
> I would certainly hope that wpa_s uses that path on the new kernel. It
> wasn't in 4.9, pretty sure.
> 
> johannes

I guess you were on the right track. For now we disabled the control_port via
driver option in the supplicant conf, because the wpa_s logs were pointing to
that and now the 4/4 frame of the 4-way handshake is encrypted as expected.

Looking to the supplicant the problematic code is in the following function.
When WPA_DRIVER_FLAGS_CONTROL_PORT is set, during rekeying at the time
of frame 4/4 the (new) PTK is not yet installed and therefore, leading to the
frame being unencrypted. The additional check for PTK being installed is probably
to prevent some race conditions. At least that's what I think after reading the
commit messages of when control port was introduced. 

int wpa_ether_send(struct wpa_supplicant *wpa_s, const u8 *dest,
		   u16 proto, const u8 *buf, size_t len)
{
[...]
	if (wpa_s->drv_flags & WPA_DRIVER_FLAGS_CONTROL_PORT) {
		int encrypt = wpa_s->wpa &&
			wpa_sm_has_ptk_installed(wpa_s->wpa);

		return wpa_drv_tx_control_port(wpa_s, dest, proto, buf, len,
					       !encrypt);
	}

	if (wpa_s->l2) {
		return l2_packet_send(wpa_s->l2, dest, proto, buf, len);
	}

	return -1;
}

Currently I don't really get the relation of PTK of the 4-way handshake and the
PEAP session key of the TLS handshake. Latter encryption should be applied
independently of the PTK being installed or not or am I wrong? Searching in
the IEEE802.11 standard didn't help me to find an answer to that question.

So I'd assume there's a bug in the wpa_supplicant or what's your impression?
Or is disabling of control_port already considered as a proper solution?

Re: wifi: mac80211: unencrypted message in 4-way handshake

[Johannes Berg]

On Tue, 2024-06-18 at 13:56 +0000, Sperling, Tobias wrote:
> > > > 
> I guess you were on the right track. For now we disabled the control_port via
> driver option in the supplicant conf, because the wpa_s logs were pointing to
> that and now the 4/4 frame of the 4-way handshake is encrypted as expected.

OK.

> Looking to the supplicant the problematic code is in the following function.
> When WPA_DRIVER_FLAGS_CONTROL_PORT is set, during rekeying at the time
> of frame 4/4 the (new) PTK is not yet installed and therefore, leading to the
> frame being unencrypted. The additional check for PTK being installed is probably
> to prevent some race conditions. At least that's what I think after reading the
> commit messages of when control port was introduced. 
> 
> int wpa_ether_send(struct wpa_supplicant *wpa_s, const u8 *dest,
> 		   u16 proto, const u8 *buf, size_t len)
> {
> [...]
> 	if (wpa_s->drv_flags & WPA_DRIVER_FLAGS_CONTROL_PORT) {
> 		int encrypt = wpa_s->wpa &&
> 			wpa_sm_has_ptk_installed(wpa_s->wpa);
> 
> 		return wpa_drv_tx_control_port(wpa_s, dest, proto, buf, len,
> 					       !encrypt);
> 	}
> 
> 	if (wpa_s->l2) {
> 		return l2_packet_send(wpa_s->l2, dest, proto, buf, len);
> 	}
> 
> 	return -1;
> }

I don't know, you should take all this to the hostap list then.

> Currently I don't really get the relation of PTK of the 4-way handshake and the
> PEAP session key of the TLS handshake. Latter encryption should be applied
> independently of the PTK being installed or not or am I wrong? Searching in
> the IEEE802.11 standard didn't help me to find an answer to that question.


The PEAP session is completely transparent to wifi as a transort - as
far as the kernel stack is concerned, the TLS handshake packets are just
data to either be encrypted with the TK (PTK) or not (depending on the
flag), but the inner encryption is totally up to the higher layer.

> So I'd assume there's a bug in the wpa_supplicant or what's your impression?

Sounds like.

> Or is disabling of control_port already considered as a proper solution?

No.

johannes

Re: wifi: mac80211: unencrypted message in 4-way handshake

[Johannes Berg]

On Tue, 2024-06-18 at 13:56 +0000, Sperling, Tobias wrote:
> > > > 
> Looking to the supplicant the problematic code is in the following function.
> When WPA_DRIVER_FLAGS_CONTROL_PORT is set, during rekeying at the time
> of frame 4/4 the (new) PTK is not yet installed and therefore, leading to the
> frame being unencrypted. The additional check for PTK being installed is probably
> to prevent some race conditions. At least that's what I think after reading the
> commit messages of when control port was introduced. 
> 
> int wpa_ether_send(struct wpa_supplicant *wpa_s, const u8 *dest,
> 		   u16 proto, const u8 *buf, size_t len)
> {
> [...]
> 	if (wpa_s->drv_flags & WPA_DRIVER_FLAGS_CONTROL_PORT) {
> 		int encrypt = wpa_s->wpa &&
> 			wpa_sm_has_ptk_installed(wpa_s->wpa);

Yeah I think the issue here might have been that this frame is queued up
for transmission, then the key is installed, and then it really is
transmitted - so then it might be encrypted when it shouldn't be.

But tracking whether the *current* PTK is installed is wrong in the
rekey case since the packets should be encrypted with the *old* PTK,
which ... will have the same race?

johannes