Linux bluetooth development
 help / color / mirror / Atom feed
* [PATCH 0/2] Bluetooth: btrtl: Don't return NULL from btrtl_initialize()
@ 2026-11-15 19:53 Artur
  2026-11-15 19:53 ` [PATCH 1/2] Bluetooth: btrtl: Return an error when reading the key id fails Artur
  2026-11-15 19:53 ` [PATCH 2/2] Bluetooth: btrtl: Set an error code when skb allocation fails Artur
  0 siblings, 2 replies; 4+ messages in thread
From: Artur @ 2026-11-15 19:53 UTC (permalink / raw)
  To: Marcel Holtmann, Luiz Augusto von Dentz
  Cc: Max Chou, linux-bluetooth, linux-kernel, Artur

btrtl_initialize() has two error paths that jump to err_free while
'ret' is still 0, so the function returns ERR_PTR(0), which is NULL.
Both callers only check the result with IS_ERR() and then dereference
the pointer.

Patch 1 fixes the path taken when reading the RTL_SEC_PROJ register
fails. It can be hit by a controller that times out or answers the
vendor command with an error during setup.

Patch 2 fixes the path taken when allocating the skb for the drop
firmware vendor command fails.

The two bugs come from different commits, so they are fixed in
separate patches with their own Fixes: tags.

Testing: compile tested only, on x86_64 against mainline commit
fa2933aac170 (after v7.4-rc2), with W=1 and sparse, without new
warnings. smatch no longer reports "passing zero to 'ERR_PTR'" in
btrtl_initialize() after patch 2. The error paths have not been
exercised at runtime: no Realtek Bluetooth controller was available
and no reproducer was written. drivers/bluetooth/btrtl.c is identical
in bluetooth-next, so the series applies there as well.

The bugs were found with smatch while checking the Bluetooth USB
drivers. The analysis, the fixes and the descriptions were prepared
with the help of an AI coding assistant (Claude Opus 5.5).

Artur (2):
  Bluetooth: btrtl: Return an error when reading the key id fails
  Bluetooth: btrtl: Set an error code when skb allocation fails

 drivers/bluetooth/btrtl.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)


base-commit: fa2933aac170dd19c6a7f0d7b81a8f537e40915c
-- 
2.43.0


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH 1/2] Bluetooth: btrtl: Return an error when reading the key id fails
  2026-11-15 19:53 [PATCH 0/2] Bluetooth: btrtl: Don't return NULL from btrtl_initialize() Artur
@ 2026-11-15 19:53 ` Artur
  2026-11-15 22:44   ` Bluetooth: btrtl: Don't return NULL from btrtl_initialize() bluez.test.bot
  2026-11-15 19:53 ` [PATCH 2/2] Bluetooth: btrtl: Set an error code when skb allocation fails Artur
  1 sibling, 1 reply; 4+ messages in thread
From: Artur @ 2026-11-15 19:53 UTC (permalink / raw)
  To: Marcel Holtmann, Luiz Augusto von Dentz
  Cc: Max Chou, linux-bluetooth, linux-kernel, Artur

btrtl_initialize() stores the result of reading the RTL_SEC_PROJ
register in the local variable 'rc', but its error path returns
ERR_PTR(ret). At that point 'ret' still holds 0 from an earlier
successful call, so a failed read makes btrtl_initialize() return
ERR_PTR(0), which is NULL.

Both callers, btrtl_setup_realtek() and h5_btrtl_setup(), only check
the result with IS_ERR(). They then pass the NULL pointer to
btrtl_download_firmware() and btrtl_get_uart_settings(), which
dereference it right away. The register is read with a vendor HCI
command, so this happens whenever the controller times out or replies
with an unexpected status or length during setup.

The read used to live in rtlbt_parse_firmware_v2(), which returned
-EIO on failure. Commit cd8dbd9ef600 ("Bluetooth: btrtl: Avoid loading
the config file on security chips") moved it into btrtl_initialize()
but kept the separate 'rc' variable.

Store the result in 'ret' so that the error reaches the caller, and
drop the now unused 'rc' variable.

The problem was found with smatch:

  drivers/bluetooth/btrtl.c:1253 btrtl_initialize() warn: passing zero to 'ERR_PTR'

The analysis, the fix and this description were prepared with the help
of an AI coding assistant (Claude Opus 5.5).

Fixes: cd8dbd9ef600 ("Bluetooth: btrtl: Avoid loading the config file on security chips")
Cc: Max Chou <max.chou@realtek.com>
Signed-off-by: Artur <arturfaleiros1254@gmail.com>
---

Notes:
    Compile tested only (x86_64, W=1, sparse, smatch). The error path has
    not been exercised at runtime: no Realtek Bluetooth controller was
    available and no reproducer was written.

 drivers/bluetooth/btrtl.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/drivers/bluetooth/btrtl.c b/drivers/bluetooth/btrtl.c
index d29813331603..d2c2e79800fc 100644
--- a/drivers/bluetooth/btrtl.c
+++ b/drivers/bluetooth/btrtl.c
@@ -1073,7 +1073,6 @@ struct btrtl_device_info *btrtl_initialize(struct hci_dev *hdev,
 	u16 hci_rev, lmp_subver;
 	u8 hci_ver, lmp_ver, chip_type = 0;
 	int ret;
-	int rc;
 	u8 key_id;
 	u8 reg_val[2];
 
@@ -1185,8 +1184,8 @@ struct btrtl_device_info *btrtl_initialize(struct hci_dev *hdev,
 		goto err_free;
 	}
 
-	rc = btrtl_vendor_read_reg16(hdev, RTL_SEC_PROJ, reg_val);
-	if (rc < 0)
+	ret = btrtl_vendor_read_reg16(hdev, RTL_SEC_PROJ, reg_val);
+	if (ret < 0)
 		goto err_free;
 
 	key_id = reg_val[0];
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 4+ messages in thread

* [PATCH 2/2] Bluetooth: btrtl: Set an error code when skb allocation fails
  2026-11-15 19:53 [PATCH 0/2] Bluetooth: btrtl: Don't return NULL from btrtl_initialize() Artur
  2026-11-15 19:53 ` [PATCH 1/2] Bluetooth: btrtl: Return an error when reading the key id fails Artur
@ 2026-11-15 19:53 ` Artur
  1 sibling, 0 replies; 4+ messages in thread
From: Artur @ 2026-11-15 19:53 UTC (permalink / raw)
  To: Marcel Holtmann, Luiz Augusto von Dentz
  Cc: Max Chou, linux-bluetooth, linux-kernel, Artur

When btrtl_initialize() asks the controller to drop its firmware and
bt_skb_alloc() fails, it jumps to its error path without setting 'ret'.
'ret' still holds 0 from the previous successful vendor command, so
btrtl_initialize() returns ERR_PTR(0), which is NULL.

Both callers, btrtl_setup_realtek() and h5_btrtl_setup(), only check
the result with IS_ERR() and then dereference the NULL pointer in
btrtl_download_firmware() and btrtl_get_uart_settings().

This path used to jump to a label that carried on with the setup.
Commit 9a24ce5e29b1 ("Bluetooth: btrtl: Firmware format v2 support")
turned it into a failure but did not set an error code.

Set 'ret' to -ENOMEM before jumping to the error path.

The problem was found with smatch, which keeps reporting the following
for this path once the RTL_SEC_PROJ read failure is fixed:

  drivers/bluetooth/btrtl.c:1252 btrtl_initialize() warn: passing zero to 'ERR_PTR'

The analysis, the fix and this description were prepared with the help
of an AI coding assistant (Claude Opus 5.5).

Fixes: 9a24ce5e29b1 ("Bluetooth: btrtl: Firmware format v2 support")
Cc: Max Chou <max.chou@realtek.com>
Signed-off-by: Artur <arturfaleiros1254@gmail.com>
---

Notes:
    Compile tested only (x86_64, W=1, sparse, smatch). The error path has
    not been exercised at runtime: no Realtek Bluetooth controller was
    available and no reproducer was written.

 drivers/bluetooth/btrtl.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/bluetooth/btrtl.c b/drivers/bluetooth/btrtl.c
index d2c2e79800fc..35a2c9d3380c 100644
--- a/drivers/bluetooth/btrtl.c
+++ b/drivers/bluetooth/btrtl.c
@@ -1143,8 +1143,10 @@ struct btrtl_device_info *btrtl_initialize(struct hci_dev *hdev,
 
 	if (btrtl_dev->drop_fw) {
 		skb = bt_skb_alloc(sizeof(*cmd), GFP_KERNEL);
-		if (!skb)
+		if (!skb) {
+			ret = -ENOMEM;
 			goto err_free;
+		}
 
 		cmd = skb_put(skb, HCI_COMMAND_HDR_SIZE);
 		cmd->opcode = cpu_to_le16(0xfc66);
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 4+ messages in thread

* RE: Bluetooth: btrtl: Don't return NULL from btrtl_initialize()
  2026-11-15 19:53 ` [PATCH 1/2] Bluetooth: btrtl: Return an error when reading the key id fails Artur
@ 2026-11-15 22:44   ` bluez.test.bot
  0 siblings, 0 replies; 4+ messages in thread
From: bluez.test.bot @ 2026-11-15 22:44 UTC (permalink / raw)
  To: linux-bluetooth, arturfaleiros1254

[-- Attachment #1: Type: text/plain, Size: 1498 bytes --]

This is automated email and please do not reply to this email!

Dear submitter,

Thank you for submitting the patches to the linux bluetooth mailing list.
This is a CI test results with your patch series:
PW Link:https://patchwork.kernel.org/series/1211959/

---Test result---

Test Summary:
CheckPatch                    PASS      0.98 seconds
VerifyFixes                   PASS      0.06 seconds
VerifySignedoff               PASS      0.07 seconds
GitLint                       FAIL      0.40 seconds
SubjectPrefix                 PASS      0.12 seconds
BuildKernel                   PASS      24.45 seconds
CheckAllWarning               PASS      27.78 seconds
CheckSparse                   PASS      30.40 seconds
BuildKernel32                 PASS      24.44 seconds
CheckKernelLLVM               PASS      24.65 seconds
TestRunnerSetup               PASS      564.82 seconds
IncrementalBuild              PASS      25.32 seconds

Details
##############################
Test: GitLint - FAIL
Desc: Run gitlint
Output:
[1/2] Bluetooth: btrtl: Return an error when reading the key id fails

26: B1 Line exceeds max length (83>80): "  drivers/bluetooth/btrtl.c:1253 btrtl_initialize() warn: passing zero to 'ERR_PTR'"
[2/2] Bluetooth: btrtl: Set an error code when skb allocation fails

21: B1 Line exceeds max length (83>80): "  drivers/bluetooth/btrtl.c:1252 btrtl_initialize() warn: passing zero to 'ERR_PTR'"


https://github.com/bluez/bluetooth-next/pull/1138

---
Regards,
Linux Bluetooth


^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2026-11-15 22:44 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-15 19:53 [PATCH 0/2] Bluetooth: btrtl: Don't return NULL from btrtl_initialize() Artur
2026-11-15 19:53 ` [PATCH 1/2] Bluetooth: btrtl: Return an error when reading the key id fails Artur
2026-11-15 22:44   ` Bluetooth: btrtl: Don't return NULL from btrtl_initialize() bluez.test.bot
2026-11-15 19:53 ` [PATCH 2/2] Bluetooth: btrtl: Set an error code when skb allocation fails Artur

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox