* [PATCH 1/2] Bluetooth: btrtl: Return an error when reading the key id fails
2026-11-15 19:53 [PATCH 0/2] Bluetooth: btrtl: Don't return NULL from btrtl_initialize() Artur
@ 2026-11-15 19:53 ` Artur
2026-11-15 22:44 ` Bluetooth: btrtl: Don't return NULL from btrtl_initialize() bluez.test.bot
2026-11-15 19:53 ` [PATCH 2/2] Bluetooth: btrtl: Set an error code when skb allocation fails Artur
1 sibling, 1 reply; 4+ messages in thread
From: Artur @ 2026-11-15 19:53 UTC (permalink / raw)
To: Marcel Holtmann, Luiz Augusto von Dentz
Cc: Max Chou, linux-bluetooth, linux-kernel, Artur
btrtl_initialize() stores the result of reading the RTL_SEC_PROJ
register in the local variable 'rc', but its error path returns
ERR_PTR(ret). At that point 'ret' still holds 0 from an earlier
successful call, so a failed read makes btrtl_initialize() return
ERR_PTR(0), which is NULL.
Both callers, btrtl_setup_realtek() and h5_btrtl_setup(), only check
the result with IS_ERR(). They then pass the NULL pointer to
btrtl_download_firmware() and btrtl_get_uart_settings(), which
dereference it right away. The register is read with a vendor HCI
command, so this happens whenever the controller times out or replies
with an unexpected status or length during setup.
The read used to live in rtlbt_parse_firmware_v2(), which returned
-EIO on failure. Commit cd8dbd9ef600 ("Bluetooth: btrtl: Avoid loading
the config file on security chips") moved it into btrtl_initialize()
but kept the separate 'rc' variable.
Store the result in 'ret' so that the error reaches the caller, and
drop the now unused 'rc' variable.
The problem was found with smatch:
drivers/bluetooth/btrtl.c:1253 btrtl_initialize() warn: passing zero to 'ERR_PTR'
The analysis, the fix and this description were prepared with the help
of an AI coding assistant (Claude Opus 5.5).
Fixes: cd8dbd9ef600 ("Bluetooth: btrtl: Avoid loading the config file on security chips")
Cc: Max Chou <max.chou@realtek.com>
Signed-off-by: Artur <arturfaleiros1254@gmail.com>
---
Notes:
Compile tested only (x86_64, W=1, sparse, smatch). The error path has
not been exercised at runtime: no Realtek Bluetooth controller was
available and no reproducer was written.
drivers/bluetooth/btrtl.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/drivers/bluetooth/btrtl.c b/drivers/bluetooth/btrtl.c
index d29813331603..d2c2e79800fc 100644
--- a/drivers/bluetooth/btrtl.c
+++ b/drivers/bluetooth/btrtl.c
@@ -1073,7 +1073,6 @@ struct btrtl_device_info *btrtl_initialize(struct hci_dev *hdev,
u16 hci_rev, lmp_subver;
u8 hci_ver, lmp_ver, chip_type = 0;
int ret;
- int rc;
u8 key_id;
u8 reg_val[2];
@@ -1185,8 +1184,8 @@ struct btrtl_device_info *btrtl_initialize(struct hci_dev *hdev,
goto err_free;
}
- rc = btrtl_vendor_read_reg16(hdev, RTL_SEC_PROJ, reg_val);
- if (rc < 0)
+ ret = btrtl_vendor_read_reg16(hdev, RTL_SEC_PROJ, reg_val);
+ if (ret < 0)
goto err_free;
key_id = reg_val[0];
--
2.43.0
^ permalink raw reply related [flat|nested] 4+ messages in thread* [PATCH 2/2] Bluetooth: btrtl: Set an error code when skb allocation fails
2026-11-15 19:53 [PATCH 0/2] Bluetooth: btrtl: Don't return NULL from btrtl_initialize() Artur
2026-11-15 19:53 ` [PATCH 1/2] Bluetooth: btrtl: Return an error when reading the key id fails Artur
@ 2026-11-15 19:53 ` Artur
1 sibling, 0 replies; 4+ messages in thread
From: Artur @ 2026-11-15 19:53 UTC (permalink / raw)
To: Marcel Holtmann, Luiz Augusto von Dentz
Cc: Max Chou, linux-bluetooth, linux-kernel, Artur
When btrtl_initialize() asks the controller to drop its firmware and
bt_skb_alloc() fails, it jumps to its error path without setting 'ret'.
'ret' still holds 0 from the previous successful vendor command, so
btrtl_initialize() returns ERR_PTR(0), which is NULL.
Both callers, btrtl_setup_realtek() and h5_btrtl_setup(), only check
the result with IS_ERR() and then dereference the NULL pointer in
btrtl_download_firmware() and btrtl_get_uart_settings().
This path used to jump to a label that carried on with the setup.
Commit 9a24ce5e29b1 ("Bluetooth: btrtl: Firmware format v2 support")
turned it into a failure but did not set an error code.
Set 'ret' to -ENOMEM before jumping to the error path.
The problem was found with smatch, which keeps reporting the following
for this path once the RTL_SEC_PROJ read failure is fixed:
drivers/bluetooth/btrtl.c:1252 btrtl_initialize() warn: passing zero to 'ERR_PTR'
The analysis, the fix and this description were prepared with the help
of an AI coding assistant (Claude Opus 5.5).
Fixes: 9a24ce5e29b1 ("Bluetooth: btrtl: Firmware format v2 support")
Cc: Max Chou <max.chou@realtek.com>
Signed-off-by: Artur <arturfaleiros1254@gmail.com>
---
Notes:
Compile tested only (x86_64, W=1, sparse, smatch). The error path has
not been exercised at runtime: no Realtek Bluetooth controller was
available and no reproducer was written.
drivers/bluetooth/btrtl.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/bluetooth/btrtl.c b/drivers/bluetooth/btrtl.c
index d2c2e79800fc..35a2c9d3380c 100644
--- a/drivers/bluetooth/btrtl.c
+++ b/drivers/bluetooth/btrtl.c
@@ -1143,8 +1143,10 @@ struct btrtl_device_info *btrtl_initialize(struct hci_dev *hdev,
if (btrtl_dev->drop_fw) {
skb = bt_skb_alloc(sizeof(*cmd), GFP_KERNEL);
- if (!skb)
+ if (!skb) {
+ ret = -ENOMEM;
goto err_free;
+ }
cmd = skb_put(skb, HCI_COMMAND_HDR_SIZE);
cmd->opcode = cpu_to_le16(0xfc66);
--
2.43.0
^ permalink raw reply related [flat|nested] 4+ messages in thread