[PATCH v2 0/4] tty/vt: fix various 512 glyph font issues

[PATCH v2 0/4] tty/vt: fix various 512 glyph font issues

[Zsolt Kajtar]

Having the 8th bit of a glyph in the attribute byte is tricky business.
These patches fix most of the cases where it went wrong in the VT code.
What remains is related to fbcon as well and so not part of this series.

This run fixes concerns raised on review and adds one more patch for a
problem which doesn't cause visible effects.

These patches were prepared to apply on tty-next.

Zsolt Kajtar (4):
  tty/vt: 8th bit location in vc_uniscr routines
  tty/vt: Prevent 8th bit corruption with soft cursor
  tty/vt: Fix unreadable kernel messages on vgacon
  tty/vt: use correct attribute mask in do_update_region

 drivers/tty/vt/vt.c | 40 ++++++++++++++++++++++++++--------------
 1 file changed, 26 insertions(+), 14 deletions(-)

-- 
2.30.2

[PATCH v2 1/4] tty/vt: 8th bit location in vc_uniscr routines

[Zsolt Kajtar]

Both vc_uniscr_check and vc_uniscr_copy_line assume that the 8th bit of
glyph is also the 8th bit in the screen buffer. However this is only the
case for fbcon at the moment. Vgacon has it on the 11th and so the
conversion won't work in that case.

The patch adds a helper to read the glyph correctly from the buffer.

Signed-off-by: Zsolt Kajtar <soci@c64.rulez.org>
---
 drivers/tty/vt/vt.c | 28 ++++++++++++++++++----------
 1 file changed, 18 insertions(+), 10 deletions(-)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 869261141..19ce9c426 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -293,6 +293,17 @@ static inline u16 *screenpos(const struct vc_data *vc, unsigned int offset,
 	return (u16 *)(origin + offset);
 }
 
+static inline u16 scr_read_glyph(const struct vc_data *vc, u16 *pos)
+{
+	u16 w = scr_readw(pos);
+	u16 glyph = w & 0xff;
+
+	if (w & vc->vc_hi_font_mask)
+		glyph |= 0x100;
+
+	return glyph;
+}
+
 static void con_putc(struct vc_data *vc, u16 ca, unsigned int y, unsigned int x)
 {
 	if (vc->vc_sw->con_putc)
@@ -493,7 +504,7 @@ int vc_uniscr_check(struct vc_data *vc)
 {
 	u32 **uni_lines;
 	unsigned short *p;
-	int x, y, mask;
+	int x, y;
 
 	WARN_CONSOLE_UNLOCKED();
 
@@ -514,11 +525,11 @@ int vc_uniscr_check(struct vc_data *vc)
 	 * unicode content will be available after a complete screen refresh.
 	 */
 	p = (unsigned short *)vc->vc_origin;
-	mask = vc->vc_hi_font_mask | 0xff;
 	for (y = 0; y < vc->vc_rows; y++) {
 		u32 *line = uni_lines[y];
 		for (x = 0; x < vc->vc_cols; x++) {
-			u16 glyph = scr_readw(p++) & mask;
+			u16 glyph = scr_read_glyph(vc, p++);
+
 			line[x] = inverse_translate(vc, glyph, true);
 		}
 	}
@@ -561,10 +572,10 @@ void vc_uniscr_copy_line(const struct vc_data *vc, void *dest, bool viewed,
 		 * buffer of its own.
 		 */
 		u16 *p = (u16 *)pos;
-		int mask = vc->vc_hi_font_mask | 0xff;
 		u32 *uni_buf = dest;
 		while (nr--) {
-			u16 glyph = scr_readw(p++) & mask;
+			u16 glyph = scr_read_glyph(vc, p++);
+
 			*uni_buf++ = inverse_translate(vc, glyph, true);
 		}
 	}
@@ -4920,12 +4931,9 @@ int con_font_op(struct vc_data *vc, struct console_font_op *op)
 /* used by selection */
 u16 screen_glyph(const struct vc_data *vc, int offset)
 {
-	u16 w = scr_readw(screenpos(vc, offset, true));
-	u16 c = w & 0xff;
+	u16 *pos = screenpos(vc, offset, true);
 
-	if (w & vc->vc_hi_font_mask)
-		c |= 0x100;
-	return c;
+	return scr_read_glyph(vc, pos);
 }
 EXPORT_SYMBOL_GPL(screen_glyph);
 
-- 
2.30.2

[PATCH v2 2/4] tty/vt: Prevent 8th bit corruption with soft cursor

[Zsolt Kajtar]

The attributes of the soft cursor are configurable and one would rightly
expect that only the attributes are to be affected. But that's not
guaranteed for a font with 512 glyphs as the 8th bit is in the attribute
byte. This patch makes sure that really only the attribute bits are
changed by the cursor and not the glyph's appearance.

Signed-off-by: Zsolt Kajtar <soci@c64.rulez.org>
---
 drivers/tty/vt/vt.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 19ce9c426..27b1afd5d 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -828,6 +828,7 @@ static void add_softcursor(struct vc_data *vc)
 {
 	int i = scr_readw((u16 *) vc->vc_pos);
 	u32 type = vc->vc_cursor_type;
+	u16 mask = vc->vc_hi_font_mask | 0xff;
 
 	if (!(type & CUR_SW))
 		return;
@@ -841,6 +842,7 @@ static void add_softcursor(struct vc_data *vc)
 		i ^= CUR_BG;
 	if ((type & CUR_INVERT_FG_BG) && (i & CUR_FG) == ((i & CUR_BG) >> 4))
 		i ^= CUR_FG;
+	i = (i & ~mask) | (softcursor_original & mask);
 	scr_writew(i, (u16 *)vc->vc_pos);
 	if (con_should_update(vc))
 		con_putc(vc, i, vc->state.y, vc->state.x);
-- 
2.30.2

[PATCH v2 3/4] tty/vt: Fix unreadable kernel messages on vgacon

[Zsolt Kajtar]

When a 512 glyph font is loaded on vgacon and the bold attributes are in
effect then the kernel console output (printk) becomes unreadable. It is
because the brightness bit (used for bold) is at the same place where
the 8th bit of the glyph index is. This patch adds the missing masking
to ensure the output will be displayed using the lower half of the font.

Signed-off-by: Zsolt Kajtar <soci@c64.rulez.org>
---
 drivers/tty/vt/vt.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 27b1afd5d..5d458211b 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -3361,7 +3361,8 @@ static void vt_console_print(struct console *co, const char *b, unsigned count)
 				continue;
 		}
 		vc_uniscr_putc(vc, c);
-		scr_writew((vc->vc_attr << 8) + c, (unsigned short *)vc->vc_pos);
+		scr_writew(((vc->vc_attr << 8) & ~vc->vc_hi_font_mask) | c,
+			   (unsigned short *)vc->vc_pos);
 		notify_write(vc, c);
 		cnt++;
 		if (vc->state.x == vc->vc_cols - 1) {
-- 
2.30.2

[PATCH v2 4/4] tty/vt: use correct attribute mask in do_update_region

[Zsolt Kajtar]

The original code assumes all bits in the attribute byte are for
attributes. If 512 glyphs are in use then one is used as the 8th bit of
the glyph. This can unnecessarily split up a region of uniform
attributes whenever that bit changes.

Signed-off-by: Zsolt Kajtar <soci@c64.rulez.org>
---
 drivers/tty/vt/vt.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 5d458211b..eb60f69f3 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -613,22 +613,23 @@ static void do_update_region(struct vc_data *vc, unsigned long start, int count)
 {
 	unsigned int xx, yy, offset;
 	u16 *p = (u16 *)start;
+	u16 mask = 0xff00 & ~vc->vc_hi_font_mask;
 
 	offset = (start - vc->vc_origin) / 2;
 	xx = offset % vc->vc_cols;
 	yy = offset / vc->vc_cols;
 
 	for(;;) {
-		u16 attrib = scr_readw(p) & 0xff00;
+		u16 attrib = scr_readw(p) & mask;
 		int startx = xx;
 		u16 *q = p;
 		while (xx < vc->vc_cols && count) {
-			if (attrib != (scr_readw(p) & 0xff00)) {
+			if (attrib != (scr_readw(p) & mask)) {
 				if (p > q)
 					vc->vc_sw->con_putcs(vc, q, p-q, yy, startx);
 				startx = xx;
 				q = p;
-				attrib = scr_readw(p) & 0xff00;
+				attrib = scr_readw(p) & mask;
 			}
 			p++;
 			xx++;
-- 
2.30.2