[PATCH] audit: fix potential integer overflow in audit_log_n_hex()

[Ricardo Robaina <rrobaina@redhat.com>]

The function audit_log_n_hex() calculates new_len as len << 1 to
account for hex encoding, where each byte becomes two characters.
However, when len is passed as size_t but new_len was declared as
int, this could lead to integer overflow for large values of len.

Additionally, the bit shift operation itself could overflow if len
exceeds SIZE_MAX / 2, leading to incorrect buffer size calculations
and potential memory corruption.

Fix this by changing new_len and loop counter i from int to size_t
to match the len parameter type, preventing signed/unsigned mismatches,
and by adding an overflow check before the bit shift operation that
calculates len << 1.

Fixes: 168b7173959f ("AUDIT: Clean up logging of untrusted strings")
Signed-off-by: Ricardo Robaina <rrobaina@redhat.com>
---
 kernel/audit.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/kernel/audit.c b/kernel/audit.c
index e1d489bc2dff..32ac50996451 100644
--- a/kernel/audit.c
+++ b/kernel/audit.c
@@ -2076,7 +2076,8 @@ void audit_log_format(struct audit_buffer *ab, const char *fmt, ...)
 void audit_log_n_hex(struct audit_buffer *ab, const unsigned char *buf,
 		size_t len)
 {
-	int i, avail, new_len;
+	int avail;
+	size_t i, new_len;
 	unsigned char *ptr;
 	struct sk_buff *skb;
 
@@ -2084,9 +2085,14 @@ void audit_log_n_hex(struct audit_buffer *ab, const unsigned char *buf,
 		return;
 
 	BUG_ON(!ab->skb);
+
+	/* prevent new_len overflow */
+	if (len > SIZE_MAX / 2)
+		return;
+
 	skb = ab->skb;
 	avail = skb_tailroom(skb);
-	new_len = len<<1;
+	new_len = len << 1;
 	if (new_len >= avail) {
 		/* Round the buffer request up to the next multiple */
 		new_len = AUDIT_BUFSIZ*(((new_len-avail)/AUDIT_BUFSIZ) + 1);
-- 
2.53.0