From: david.laight.linux@gmail.com
To: linux-kernel@vger.kernel.org, linux-serial@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
Jiri Slaby <jirislaby@kernel.org>,
David Laight <david.laight.linux@gmail.com>
Subject: [PATCH 27/44] drivers/tty/vt: use umin() instead of min_t(u16, ...) for row/col limits
Date: Wed, 19 Nov 2025 22:41:23 +0000 [thread overview]
Message-ID: <20251119224140.8616-28-david.laight.linux@gmail.com> (raw)
In-Reply-To: <20251119224140.8616-1-david.laight.linux@gmail.com>
From: David Laight <david.laight.linux@gmail.com>
The row/column bounds (for a screen window box) are changed from
'offset one' to 'offset zero' and bound to the screen size using:
v->xs = min_t(u16, v->xs - 1, vc->vc_cols - 1);
This has the side effect of converting zero to the limit.
A check I'm adding to min_t() reports that (u16)(v->xs - 1) (etc)
discards signiticant bits (because v->xs is promoted to 'int' before
the addition).
If v->xs is zero (it comes from userspace) it converts -1 to 0xffff.
This is then bounded to 'vc->vc_cols - 1' which will be fine.
Replace with:
v->xs = umin(v->xs - 1, vc->vc_cols - 1);
which again converts a -1 to unsigned - this time to 0xffffffff,
with the same overall effect.
Whether zero is meant to mean the 'maximum size' is unknown.
I can't find any documentation for the ioctl and it pre-dates git.
Detected by an extra check added to min_t().
Signed-off-by: David Laight <david.laight.linux@gmail.com>
---
drivers/tty/vt/selection.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/drivers/tty/vt/selection.c b/drivers/tty/vt/selection.c
index 07d3b93975d3..13f4e48b4142 100644
--- a/drivers/tty/vt/selection.c
+++ b/drivers/tty/vt/selection.c
@@ -348,10 +348,11 @@ static int vc_selection(struct vc_data *vc, struct tiocl_selection *v,
return 0;
}
- v->xs = min_t(u16, v->xs - 1, vc->vc_cols - 1);
- v->ys = min_t(u16, v->ys - 1, vc->vc_rows - 1);
- v->xe = min_t(u16, v->xe - 1, vc->vc_cols - 1);
- v->ye = min_t(u16, v->ye - 1, vc->vc_rows - 1);
+ /* Historically 0 => max value */
+ v->xs = umin(v->xs - 1, vc->vc_cols - 1);
+ v->ys = umin(v->ys - 1, vc->vc_rows - 1);
+ v->xe = umin(v->xe - 1, vc->vc_cols - 1);
+ v->ye = umin(v->ye - 1, vc->vc_rows - 1);
if (mouse_reporting() && (v->sel_mode & TIOCL_SELMOUSEREPORT)) {
mouse_report(tty, v->sel_mode & TIOCL_SELBUTTONMASK, v->xs,
--
2.39.5
next prev parent reply other threads:[~2025-11-19 22:42 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-11-19 22:40 [PATCH 00/44] Change a lot of min_t() that might mask high bits david.laight.linux
2025-11-19 22:41 ` david.laight.linux [this message]
2025-11-20 7:23 ` [PATCH 27/44] drivers/tty/vt: use umin() instead of min_t(u16, ...) for row/col limits Jiri Slaby
2025-11-20 1:47 ` [PATCH 00/44] Change a lot of min_t() that might mask high bits Jakub Kicinski
2025-11-20 9:38 ` Lorenzo Stoakes
2025-11-20 14:52 ` (subset) " Jens Axboe
2025-11-24 9:49 ` Herbert Xu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20251119224140.8616-28-david.laight.linux@gmail.com \
--to=david.laight.linux@gmail.com \
--cc=gregkh@linuxfoundation.org \
--cc=jirislaby@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-serial@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox