[PATCH bluetooth-next] Bluetooth: hci_event: strip Broadcom vendor bits from legacy adv report type

[Ton Sharp <66ton99@gmail.com>]

Broadcom controllers running ROM firmware on Apple T2 Macs (where no
RAM patchram exists, e.g. BCM4364B3 on MacBookPro16,1) set vendor bits
0x10/0x20/0x80 in the legacy LE Advertising Report Event_Type.

As a result, directed advertisements (ADV_DIRECT_IND) arrive as
0x81/0x91/0xa1 instead of 0x01. Because process_adv_report() drops any
legacy report with type > LE_ADV_SCAN_RSP (0x04) as an unknown advertising
packet type, bonded LE peripherals (such as Apple Magic Mouse/Keyboard)
and Chrome passkey proximity scans fail to discover or reliably reconnect
to the host.

The valid legacy type is always located in the low nibble. Mask the type
field for Broadcom controllers (manufacturer 15) when it is out of range.
Valid legacy reports (0x00-0x04) remain untouched, low nibbles >= 5 remain
out of range and will continue to be dropped, and controllers from other
manufacturers are unaffected.

The fix is based on the initial t2linux patch from Jose Miguel Ochoa.

Link: https://lore.kernel.org/r/3352A23E-EA42-4366-BBAA-459CEAE6F51B@live.com
Suggested-by: Jose Miguel Ochoa <josemiguelo.ochoa@gmail.com>
Signed-off-by: Ton Sharp <66ton99@gmail.com>
---
 net/bluetooth/hci_event.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/net/bluetooth/hci_event.c b/net/bluetooth/hci_event.c
index c055d16cfb8e..a03a33bca095 100644
--- a/net/bluetooth/hci_event.c
+++ b/net/bluetooth/hci_event.c
@@ -6593,6 +6593,17 @@ static void hci_le_adv_report_evt(struct hci_dev *hdev, void *data,
 
 		hci_store_wake_reason(hdev, &info->bdaddr, info->bdaddr_type);
 
+		/* Broadcom ROM firmware (BCM4364B3 on Apple T2 Macs, no
+		 * patchram) sets vendor bits in the legacy Event_Type.
+		 */
+		if (hdev->manufacturer == 15 && info->type > LE_ADV_SCAN_RSP) {
+			/* 0x0f masks out upper vendor bits (0x10, 0x20, 0x80),
+			 * extracting the standard legacy advertising type
+			 * (0x00-0x04) stored in the low nibble (bits 0-3).
+			 */
+			info->type &= 0x0f;
+		}
+
 		if (info->length <= max_adv_len(hdev)) {
 			rssi = info->data[info->length];
 			process_adv_report(hdev, info->type, &info->bdaddr,
-- 
2.52.0