[RFC PATCH 0/2] Bluetooth: detect broken extended scan instead of guessing by chip id

[RFC PATCH 0/2] Bluetooth: detect broken extended scan instead of guessing by chip id

[Kamil Serwus]

Hi,

commit 5ead2063611ae5 ("Bluetooth: btrtl: fix RTL8761B/BU broken LE
extended scan"), which landed in v7.2, regresses a different RTL8761BU
dongle than the one it was written for. This series reports that and
proposes a way out that should not need a new patch per dongle.

The regression
--------------

Hardware here is USB 0bda:8771, firmware 0xdfc6d922, ROM lmp_subver
0x8761 - an RTL8761BU, so CHIP_ID_8761B, so it gets the new quirk, which
makes use_ext_scan() false and moves scanning from LE Set Extended Scan
Enable (0x2042) to the legacy LE Set Scan Enable (0x200c).

This dongle is the opposite of the one the quirk was written for:
extended scan works flawlessly, but the firmware stops answering the
legacy scan-disable command about ten seconds into any scan:

  Bluetooth: hci0: command 0x200c tx timeout
  Bluetooth: hci0: Opcode 0x200c failed: -110
  Bluetooth: hci0: Unable to disable scanning: -110
  Bluetooth: hci0: disable scanning failed: -110
  Bluetooth: hci0: Resetting usb device.
  Bluetooth: hci0: start background scanning failed: -110
  usb 3-1: reset full-speed USB device number 2 using xhci_hcd

btusb resets the device, it re-enumerates (hci0 -> hci1 -> hci2 -> ...)
and every connected BLE device is dropped. With a Bluetooth keyboard and
trackball that leaves the machine with no input at all.

v7.1.7 drives the same dongle, with byte-identical linux-firmware and
identical CONFIG_BT_HCIBTUSB_*, for eleven hours and nine BLE HID
connects without a single scan error.

Why not just narrow the quirk
-----------------------------

Nothing available at setup time separates the two dongles:

  - same project_id (CHIP_ID_8761B)
  - same ic_id_table entry, IC_INFO(RTL_ROM_LMP_8761A, 0xb, 0xa, HCI_USB)
  - same firmware file, rtl_bt/rtl8761bu_fw.bin
  - both advertise the extended scan commands in commands[37]

They differ only by USB id, which btrtl_set_quirks() cannot see.
driver_info cannot carry it either: 0bda:a728 has no entry of its own, so
it falls to the generic Realtek entry in btusb's quirks_table -
USB_VENDOR_AND_INTERFACE_INFO(0x0bda, 0xe0, 0x01, 0x01) - which precedes
the per-device entries. usb_match_id() takes the first match and
0bda:8771 also presents e0/01/01, so the specific entry for it at
quirks_table:840 never applies.

What this series does instead
-----------------------------

The quirk is documented as being for controllers which "erroneously
claim to support extended scanning" - and one that claims support in
commands[37] and then answers Command Disallowed is saying so itself.
Patch 1 latches the quirk right where that command is issued; every
later scan uses the legacy commands. Patch 2 then drops the static
per-chip-id guess, which 0bda:a728 no longer needs. In this order the
series is bisectable: patch 1 alone changes nothing for anyone whose
quirk is already set at setup.

An affected controller pays one rejected scan round per power cycle
instead of a permanent stream of failures, and dongles nobody has owned
yet are handled without waiting for a patch.

The latch is sticky, as quirk_flags is never cleared, so it matters that
false positives are unlikely: hci_passive_scan_sync() always disables
scanning before starting it, so the controller is never asked to enable
a scan that is already running - the obvious legitimate source of
Command Disallowed.

Testing
-------

Built and running on v7.2 on the 0bda:8771 dongle. Extended scan is used,
the quirk is never set, no 0x200c and no controller resets, and a
keyboard plus a trackball stay connected across repeated LE scans - the
exact sequence that wedged the adapter within ten seconds before.

What I cannot test is the other half: I do not have a 0bda:a728, so the
err == -EBUSY path has never executed here. Alexej, does your dongle
still work with this series, and do you see the "extended scan rejected,
using legacy scan" warning once? Also, 0x2042 carries both the enable and
the disable, so I cannot tell from your report which call site returned
-EBUSY - knowing that would tell us whether a single rejected round is
really all it costs you.

Sending as RFC mainly because patch 1 sets the quirk at runtime, while
the existing comment says it is set before hci_register_dev or in
hdev->setup. I updated that comment, but if you would rather keep the
quirk strictly setup-time I am happy to respin as a USB-id match in
btusb.c instead - it just needs the quirks_table ordering sorted out
first.

#regzbot introduced: 5ead2063611ae56809b1b113ac44cef9547c81d7

Kamil Serwus (2):
  Bluetooth: hci_sync: latch broken ext scan on Command Disallowed
  Bluetooth: btrtl: drop the blanket RTL8761B extended scan quirk

 drivers/bluetooth/btrtl.c   | 13 -------------
 include/net/bluetooth/hci.h |  4 ++--
 net/bluetooth/hci_sync.c    | 15 +++++++++++++--
 3 files changed, 15 insertions(+), 17 deletions(-)


base-commit: 8d3ae59288f1e7d58d76558a6ee96d533bc5019f
-- 
2.55.0

[RFC PATCH 1/2] Bluetooth: hci_sync: latch broken ext scan on Command Disallowed

[Kamil Serwus]

HCI_QUIRK_BROKEN_EXT_SCAN exists for controllers that "erroneously claim
to support extended scanning". Today every user of it has to be
recognised up front - by chip id or USB id - which means a controller is
only handled once somebody has owned one, hit the bug and sent a patch.

Recognising them by behaviour instead is both simpler and complete: a
controller that advertises the extended scan commands in its supported
commands bitmap but answers Command Disallowed is exactly the class of
device the quirk describes, and it says so itself on the first attempt.

Latch the quirk there, so every later scan uses the legacy commands.
The cost for an affected controller is one rejected scan round per power
cycle instead of a permanent stream of failures; for everyone else
nothing changes.

Signed-off-by: Kamil Serwus <kserwus@gmail.com>
---
 include/net/bluetooth/hci.h |  4 ++--
 net/bluetooth/hci_sync.c    | 15 +++++++++++++--
 2 files changed, 15 insertions(+), 4 deletions(-)

diff --git a/include/net/bluetooth/hci.h b/include/net/bluetooth/hci.h
index 50f0eef71..5cee010f2 100644
--- a/include/net/bluetooth/hci.h
+++ b/include/net/bluetooth/hci.h
@@ -276,8 +276,8 @@ enum {
 	 * disabled. This is required for some Broadcom controllers which
 	 * erroneously claim to support extended scanning.
 	 *
-	 * This quirk can be set before hci_register_dev is called or
-	 * during the hdev->setup vendor callback.
+	 * This quirk can be set before hci_register_dev is called, during the
+	 * hdev->setup vendor callback, or at runtime on Command Disallowed.
 	 */
 	HCI_QUIRK_BROKEN_EXT_SCAN,
 
diff --git a/net/bluetooth/hci_sync.c b/net/bluetooth/hci_sync.c
index c8d14128c..eb711f454 100644
--- a/net/bluetooth/hci_sync.c
+++ b/net/bluetooth/hci_sync.c
@@ -2265,6 +2265,7 @@ static int hci_le_set_ext_scan_enable_sync(struct hci_dev *hdev, u8 val,
 					   u8 filter_dup)
 {
 	struct hci_cp_le_set_ext_scan_enable cp;
+	int err;
 
 	memset(&cp, 0, sizeof(cp));
 	cp.enable = val;
@@ -2274,8 +2275,18 @@ static int hci_le_set_ext_scan_enable_sync(struct hci_dev *hdev, u8 val,
 	else
 		cp.filter_dup = filter_dup;
 
-	return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_EXT_SCAN_ENABLE,
-				     sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+	err = __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_EXT_SCAN_ENABLE,
+				    sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+
+	/* The controller claims extended scan support but rejects it. Latch
+	 * the quirk so that every later scan uses the legacy commands.
+	 */
+	if (err == -EBUSY) {
+		bt_dev_warn(hdev, "extended scan rejected, using legacy scan");
+		hci_set_quirk(hdev, HCI_QUIRK_BROKEN_EXT_SCAN);
+	}
+
+	return err;
 }
 
 static int hci_le_set_scan_enable_sync(struct hci_dev *hdev, u8 val,
-- 
2.55.0

[RFC PATCH 2/2] Bluetooth: btrtl: drop the blanket RTL8761B extended scan quirk

[Kamil Serwus]

Commit 5ead2063611ae5 ("Bluetooth: btrtl: fix RTL8761B/BU broken LE
extended scan") sets HCI_QUIRK_BROKEN_EXT_SCAN for every CHIP_ID_8761B.
It was validated on one dongle, USB 0bda:a728, and it regresses others
that share the chip id but not the firmware bug.

On USB 0bda:8771 (fw 0xdfc6d922) extended scan works flawlessly, while
the legacy scan-disable command the quirk forces makes the controller
stop answering entirely:

  Bluetooth: hci0: command 0x200c tx timeout
  Bluetooth: hci0: Opcode 0x200c failed: -110
  Bluetooth: hci0: Unable to disable scanning: -110
  Bluetooth: hci0: Resetting usb device.

btusb then resets the device, it re-enumerates (hci0 -> hci1 -> hci2)
and every connected BLE device is dropped - here a Kensington SlimBlade
Pro trackball and a Keychron Q8 Max keyboard. Linux 7.1.x, without the
quirk, drives the same dongle with the same firmware for hours without a
single scan error.

Nothing in btrtl can tell the two apart: same project_id, same
ic_id_table entry, same rtl8761bu_fw.bin. They differ only by USB id,
which btrtl_set_quirks() cannot see, and both take their driver_info
from the same generic Realtek entry in btusb's quirks_table, so
driver_info cannot separate them either.

With the previous patch the 0bda:a728 dongle recognises itself on its
first rejected extended scan, so the static guess is no longer needed.

Fixes: 5ead2063611ae ("Bluetooth: btrtl: fix RTL8761B/BU broken LE extended scan")
Signed-off-by: Kamil Serwus <kserwus@gmail.com>
---
 drivers/bluetooth/btrtl.c | 13 -------------
 1 file changed, 13 deletions(-)

diff --git a/drivers/bluetooth/btrtl.c b/drivers/bluetooth/btrtl.c
index 7f54d2d2d..03fa9409e 100644
--- a/drivers/bluetooth/btrtl.c
+++ b/drivers/bluetooth/btrtl.c
@@ -1343,19 +1343,6 @@ void btrtl_set_quirks(struct hci_dev *hdev, struct btrtl_device_info *btrtl_dev)
 	if (!btrtl_dev->ic_info)
 		return;
 
-	switch (btrtl_dev->project_id) {
-	case CHIP_ID_8761B:
-		/* RTL8761B/BU reports HCI version 5.1 but does not support
-		 * the LE Extended Scan commands (Opcode 0x2042), causing
-		 * repeated -EBUSY failures when BlueZ attempts extended
-		 * scanning while a connection is active.
-		 */
-		hci_set_quirk(hdev, HCI_QUIRK_BROKEN_EXT_SCAN);
-		break;
-	default:
-		break;
-	}
-
 	switch (btrtl_dev->ic_info->lmp_subver) {
 	case RTL_ROM_LMP_8703B:
 		/* 8723CS reports two pages for local ext features,
-- 
2.55.0

Re: [RFC PATCH 0/2] Bluetooth: detect broken extended scan instead of guessing by chip id

[Thorsten Leemhuis]

Hi! Thx for the report.

On 8/22/26 16:11, Kamil Serwus wrote:
> commit 5ead2063611ae5 ("Bluetooth: btrtl: fix RTL8761B/BU broken LE
> extended scan"), which landed in v7.2,

And a few days ago made it to 7.1.9 and 6.18.45. :-/

> regresses a different RTL8761BU
> dongle than the one it was written for. This series reports that and
> proposes a way out that should not need a new patch per dongle.

Not my area of expertise, so the bluetooth maintainers might disagree
with me, but FWIW:

Given the above and the fact that we are in the merge window right now
I'd say the better approach would be to fix this fast by reverting the
culprit in mainline (then stable can pick up the revert to fix the
problem there) to buy time to work this out properly.

Would you maybe be willing to submit a revert?

Ciao, Thorsten
> The regression
> --------------
> 
> Hardware here is USB 0bda:8771, firmware 0xdfc6d922, ROM lmp_subver
> 0x8761 - an RTL8761BU, so CHIP_ID_8761B, so it gets the new quirk, which
> makes use_ext_scan() false and moves scanning from LE Set Extended Scan
> Enable (0x2042) to the legacy LE Set Scan Enable (0x200c).
> 
> This dongle is the opposite of the one the quirk was written for:
> extended scan works flawlessly, but the firmware stops answering the
> legacy scan-disable command about ten seconds into any scan:
> 
>   Bluetooth: hci0: command 0x200c tx timeout
>   Bluetooth: hci0: Opcode 0x200c failed: -110
>   Bluetooth: hci0: Unable to disable scanning: -110
>   Bluetooth: hci0: disable scanning failed: -110
>   Bluetooth: hci0: Resetting usb device.
>   Bluetooth: hci0: start background scanning failed: -110
>   usb 3-1: reset full-speed USB device number 2 using xhci_hcd
> 
> btusb resets the device, it re-enumerates (hci0 -> hci1 -> hci2 -> ...)
> and every connected BLE device is dropped. With a Bluetooth keyboard and
> trackball that leaves the machine with no input at all.
> 
> v7.1.7 drives the same dongle, with byte-identical linux-firmware and
> identical CONFIG_BT_HCIBTUSB_*, for eleven hours and nine BLE HID
> connects without a single scan error.
> 
> Why not just narrow the quirk
> -----------------------------
> 
> Nothing available at setup time separates the two dongles:
> 
>   - same project_id (CHIP_ID_8761B)
>   - same ic_id_table entry, IC_INFO(RTL_ROM_LMP_8761A, 0xb, 0xa, HCI_USB)
>   - same firmware file, rtl_bt/rtl8761bu_fw.bin
>   - both advertise the extended scan commands in commands[37]
> 
> They differ only by USB id, which btrtl_set_quirks() cannot see.
> driver_info cannot carry it either: 0bda:a728 has no entry of its own, so
> it falls to the generic Realtek entry in btusb's quirks_table -
> USB_VENDOR_AND_INTERFACE_INFO(0x0bda, 0xe0, 0x01, 0x01) - which precedes
> the per-device entries. usb_match_id() takes the first match and
> 0bda:8771 also presents e0/01/01, so the specific entry for it at
> quirks_table:840 never applies.
> 
> What this series does instead
> -----------------------------
> 
> The quirk is documented as being for controllers which "erroneously
> claim to support extended scanning" - and one that claims support in
> commands[37] and then answers Command Disallowed is saying so itself.
> Patch 1 latches the quirk right where that command is issued; every
> later scan uses the legacy commands. Patch 2 then drops the static
> per-chip-id guess, which 0bda:a728 no longer needs. In this order the
> series is bisectable: patch 1 alone changes nothing for anyone whose
> quirk is already set at setup.
> 
> An affected controller pays one rejected scan round per power cycle
> instead of a permanent stream of failures, and dongles nobody has owned
> yet are handled without waiting for a patch.
> 
> The latch is sticky, as quirk_flags is never cleared, so it matters that
> false positives are unlikely: hci_passive_scan_sync() always disables
> scanning before starting it, so the controller is never asked to enable
> a scan that is already running - the obvious legitimate source of
> Command Disallowed.
> 
> Testing
> -------
> 
> Built and running on v7.2 on the 0bda:8771 dongle. Extended scan is used,
> the quirk is never set, no 0x200c and no controller resets, and a
> keyboard plus a trackball stay connected across repeated LE scans - the
> exact sequence that wedged the adapter within ten seconds before.
> 
> What I cannot test is the other half: I do not have a 0bda:a728, so the
> err == -EBUSY path has never executed here. Alexej, does your dongle
> still work with this series, and do you see the "extended scan rejected,
> using legacy scan" warning once? Also, 0x2042 carries both the enable and
> the disable, so I cannot tell from your report which call site returned
> -EBUSY - knowing that would tell us whether a single rejected round is
> really all it costs you.
> 
> Sending as RFC mainly because patch 1 sets the quirk at runtime, while
> the existing comment says it is set before hci_register_dev or in
> hdev->setup. I updated that comment, but if you would rather keep the
> quirk strictly setup-time I am happy to respin as a USB-id match in
> btusb.c instead - it just needs the quirks_table ordering sorted out
> first.
> 
> #regzbot introduced: 5ead2063611ae56809b1b113ac44cef9547c81d7
> 
> Kamil Serwus (2):
>   Bluetooth: hci_sync: latch broken ext scan on Command Disallowed
>   Bluetooth: btrtl: drop the blanket RTL8761B extended scan quirk
> 
>  drivers/bluetooth/btrtl.c   | 13 -------------
>  include/net/bluetooth/hci.h |  4 ++--
>  net/bluetooth/hci_sync.c    | 15 +++++++++++++--
>  3 files changed, 15 insertions(+), 17 deletions(-)
> 
> 
> base-commit: 8d3ae59288f1e7d58d76558a6ee96d533bc5019f

Re: [RFC PATCH 0/2] Bluetooth: detect broken extended scan instead of guessing by chip id

[Kamil Serwus]

Hi Thorsten,

> Would you maybe be willing to submit a revert?

Yes, sending it now with Cc: stable so it can reach 7.1.y and 6.18.y.

It does put Alexej's dongle (0bda:a728) back to logging repeated -EBUSY
on extended scan - noisy, but usable, unlike the current state on mine.
The RFC series stays on the list for the proper fix; Alexej, testing it
would still be welcome.

Kamil