[PATCH AUTOSEL 6.17-5.4] media: redrat3: use int type to store negative error codes

[Sasha Levin <sashal@kernel.org>]

From: Qianfeng Rong <rongqianfeng@vivo.com>

[ Upstream commit ecba852dc9f4993f4f894ea1f352564560e19a3e ]

Change "ret" from u8 to int type in redrat3_enable_detector() to store
negative error codes or zero returned by redrat3_send_cmd() and
usb_submit_urb() - this better aligns with the coding standards and
maintains code consistency.

No effect on runtime.

Signed-off-by: Qianfeng Rong <rongqianfeng@vivo.com>
Signed-off-by: Sean Young <sean@mess.org>
Signed-off-by: Hans Verkuil <hverkuil+cisco@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---

LLM Generated explanations, may be completely bogus:

YES

- What it fixes
  - Prevents truncation of negative error codes in
    `redrat3_enable_detector()`. Previously `ret` was `u8`, so failures
    from `usb_submit_urb()` (negative errno) would wrap to a positive
    byte (e.g., `-EPIPE` → `224`), causing the caller to treat failures
    as success.
  - The caller checks only for negative returns: `retval =
    redrat3_enable_detector(rr3); if (retval < 0) ...` in
    `drivers/media/rc/redrat3.c:1114-1116`. With a `u8` `ret`, errors
    would be lost, the probe would continue, and the device could end up
    non-functional (URBs not running) while the driver reports success.

- Evidence in code
  - `redrat3_send_cmd()` returns negative errno or non-negative status;
    it’s already `int`: `drivers/media/rc/redrat3.c:394-419`.
  - The patch changes `ret` to `int` in the detector enable path:
    `drivers/media/rc/redrat3.c:425`.
  - URB submissions return negative errno on error; these are assigned
    to and returned via `ret`: `drivers/media/rc/redrat3.c:439-443`,
    `drivers/media/rc/redrat3.c:445-449`. With `ret` as `u8`, a negative
    error like `-EPIPE` becomes a large positive and bypasses the `< 0`
    check at the call site (`drivers/media/rc/redrat3.c:1114-1116`).

- Scope and risk
  - Minimal, localized change (1 line, one function, single driver).
  - No API/ABI change; no behavior change on success paths; only
    corrects error propagation.
  - Aligns with kernel conventions where error codes are negative
    `int`s; the rest of this driver already uses `int ret` broadly
    (e.g., `drivers/media/rc/redrat3.c:503`,
    `drivers/media/rc/redrat3.c:657`, `drivers/media/rc/redrat3.c:691`,
    etc.), improving consistency.

- Stable backport criteria
  - Fixes a real, user-visible bug (driver may “succeed” initialization
    while URBs failed, leading to non-working IR receive).
  - Small and contained; no architectural change; regression risk is
    very low.
  - Not a feature addition; strictly a correctness fix for error
    handling.

Given the clear correctness improvement, minimal risk, and user impact
on failure paths, this commit is a good candidate for stable backport.

 drivers/media/rc/redrat3.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/media/rc/redrat3.c b/drivers/media/rc/redrat3.c
index d89a4cfe3c895..a49173f54a4d0 100644
--- a/drivers/media/rc/redrat3.c
+++ b/drivers/media/rc/redrat3.c
@@ -422,7 +422,7 @@ static int redrat3_send_cmd(int cmd, struct redrat3_dev *rr3)
 static int redrat3_enable_detector(struct redrat3_dev *rr3)
 {
 	struct device *dev = rr3->dev;
-	u8 ret;
+	int ret;
 
 	ret = redrat3_send_cmd(RR3_RC_DET_ENABLE, rr3);
 	if (ret != 0)
-- 
2.51.0