[PATCH v2 0/5] ui/console-vc: various fixes and improvements

[PATCH v2 0/5] ui/console-vc: various fixes and improvements

[Roman Penyaev]

This patch series brings several fixes and improvements to the QEMU
virtual console (VC), making it more usable and aligning it better
with VT100 spec. The updates include support for inserting and
deleting characters, better cursor handling and correct position
reporting. Without these changes, the console is hardly usable.

1. Support for DCH (delete) and ICH (insert) commands
  * Properly handles character deletion and insertion as per VT100
    specs.

2. Add support for cursor save/restore (ESC 7 and ESC 8)
  * Implements legacy DEC sequences for compatibility.
  * Ensures cursor attributes are also saved and restored.

3. Fix cursor position reporting
  * Reports cursor position relative to the screen instead of the
    scroll buffer.
  * Fixes issues with tools that rely on accurate cursor placement.

4. Ensure DSR (Device Status Report) responses go to applications
  * Stops terminal from rendering responses to the screen but instead
    send reports back to applications.

5. Handle `ESC ( <ch>` character set sequences
  * Makes `top` output look clean by handling the character set
    commands.

These changes improve the virtual console's compatibility with
terminal-based apps. Without them, the console is quite frustrating to
use.

v1 .. v2:

* The VT spec and ECMA-48 standard are somewhat vague about exactly
  what attributes should be set for the ICH (insert character),
  but various terminal implementations reset the character to the
  default state. Instead of inserting a "space" with the current
  attribute, call `vc_clear_xy()` for the inserted character.

Signed-off-by: Roman Penyaev <r.peniaev@gmail.com>
Cc: "Marc-André Lureau" <marcandre.lureau@redhat.com>
Cc: Kevin Wolf <kwolf@redhat.com>
Cc: Daniel P. Berrange <berrange@redhat.com>
Cc: qemu-devel@nongnu.org

Roman Penyaev (5):
  ui/console-vc: introduce parsing of the 'ESC ( <ch>' sequence
  ui/console-vc: report to the application instead of screen rendering
  ui/console-vc: report cursor position in the screen not in the scroll
    buffer
  ui/console-vc: add support for cursor DECSC and DECRC commands
  ui/console-vc: implement DCH (delete) and ICH (insert) commands

 ui/console-vc.c | 154 ++++++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 142 insertions(+), 12 deletions(-)

-- 
2.43.0

[PATCH v2 1/5] ui/console-vc: introduce parsing of the 'ESC ( <ch>' sequence

[Roman Penyaev]

This change introduces parsing of the 'ESC ( <ch>' sequence, which is
supposed to change character set [1]. In the QEMU case, the
introduced parsing logic does not actually change the character set, but
simply parses the sequence and does not let output of a tool to be
corrupted with leftovers: `top` sends 'ESC ( B', so if character
sequence is not parsed correctly, chracter 'B' appears in the output:

  Btop - 11:08:42 up 5 min,  1 user,  load average: 0BB
  Tasks:B 158 Btotal,B  1 Brunning,B 157 Bsleeping,B   0 BsBB
  %Cpu(s):B  0.0 Bus,B  0.0 Bsy,B  0.0 Bni,B 99.8 Bid,B  0.2 BB
  MiB Mem :B   7955.6 Btotal,B   7778.6 Bfree,B     79.6 BB
  MiB Swap:B      0.0 Btotal,B      0.0 Bfree,B      0.0 BB

      PID USER      PR  NI    VIRT    RES    SHR S B
  B    735 root      20   0    9328   3540   3152 R B
  B      1 root      20   0   20084  10904   8404 S B
  B      2 root      20   0       0      0      0 S B

[1] https://vt100.net/docs/vt100-ug/chapter3.html#SCS

Signed-off-by: Roman Penyaev <r.peniaev@gmail.com>
Cc: "Marc-André Lureau" <marcandre.lureau@redhat.com>
Cc: qemu-devel@nongnu.org
---
 ui/console-vc.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/ui/console-vc.c b/ui/console-vc.c
index fe20579832a5..90ff0ffda8c5 100644
--- a/ui/console-vc.c
+++ b/ui/console-vc.c
@@ -42,6 +42,8 @@ enum TTYState {
     TTY_STATE_NORM,
     TTY_STATE_ESC,
     TTY_STATE_CSI,
+    TTY_STATE_G0,
+    TTY_STATE_G1,
 };
 
 typedef struct QemuTextConsole {
@@ -694,6 +696,10 @@ static void vc_putchar(VCChardev *vc, int ch)
                 vc->esc_params[i] = 0;
             vc->nb_esc_params = 0;
             vc->state = TTY_STATE_CSI;
+        } else if (ch == '(') {
+            vc->state = TTY_STATE_G0;
+        } else if (ch == ')') {
+            vc->state = TTY_STATE_G1;
         } else {
             vc->state = TTY_STATE_NORM;
         }
@@ -844,6 +850,16 @@ static void vc_putchar(VCChardev *vc, int ch)
             }
             break;
         }
+        break;
+    case TTY_STATE_G0: /* set character sets */
+    case TTY_STATE_G1: /* set character sets */
+        switch (ch) {
+        case 'B':
+            /* Latin-1 map */
+            break;
+        }
+        vc->state = TTY_STATE_NORM;
+        break;
     }
 }
 
-- 
2.43.0

[PATCH v2 2/5] ui/console-vc: report to the application instead of screen rendering

[Roman Penyaev]

Terminal Device Status Report (DSR) [1] should be sent to an
application, not rendered to the screen. This patch fixes rendering of
terminal report, which appear only on the graphical screen of the
terminal (console "vc") and can be reproduced by the following
command:

     echo -en '\e[6n'; IFS='[;' read -sdR _ row col; echo $row:$col

Command requests cursor position and waits for terminal response, but
instead, the response is rendered to the graphical screen and never
sent to an application.

Why bother? Busybox shell (ash) in Alpine distribution requests cursor
position on each shell prompt (once <ENTER> is pressed), which makes a
prompt on a graphical screen corrupted with repeating Cursor Position
Report (CPR) [2]:

   [root@alpine ~]# \033[57;1R]

Which is very annoying and incorrect.

[1] https://vt100.net/docs/vt100-ug/chapter3.html#DSR
[2] https://vt100.net/docs/vt100-ug/chapter3.html#CPR

Signed-off-by: Roman Penyaev <r.peniaev@gmail.com>
Cc: "Marc-André Lureau" <marcandre.lureau@redhat.com>
Cc: qemu-devel@nongnu.org
---
 ui/console-vc.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/ui/console-vc.c b/ui/console-vc.c
index 90ff0ffda8c5..d512f57e10a9 100644
--- a/ui/console-vc.c
+++ b/ui/console-vc.c
@@ -617,10 +617,9 @@ static void vc_put_one(VCChardev *vc, int ch)
 
 static void vc_respond_str(VCChardev *vc, const char *buf)
 {
-    while (*buf) {
-        vc_put_one(vc, *buf);
-        buf++;
-    }
+    QemuTextConsole *s = vc->console;
+
+    qemu_chr_be_write(s->chr, (const uint8_t *)buf, strlen(buf));
 }
 
 /* set cursor, checking bounds */
-- 
2.43.0

[PATCH v2 3/5] ui/console-vc: report cursor position in the screen not in the scroll buffer

[Roman Penyaev]

The format of the CSI cursor position report is `ESC[row;columnR`,
where `row` is a row of a cursor in the screen, not in the scrollback
buffer. What's the difference? Let's say the terminal screen has 24
lines, no matter how long the scrollback buffer may be, the last line
is the 24th.

For example the following command can be executed in xterm on the last
screen line:

   $ echo -en '\e[6n'; IFS='[;' read -sdR _ row col; echo $row:$col
   24:1

It shows the cursor position on the current screen and not relative
to the backscroll buffer.

Before this change the row number was always increasing for the QEMU
VC and represents the cursor position relative to the backscroll
buffer.

Signed-off-by: Roman Penyaev <r.peniaev@gmail.com>
Cc: "Marc-André Lureau" <marcandre.lureau@redhat.com>
Cc: qemu-devel@nongnu.org
---
 ui/console-vc.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/ui/console-vc.c b/ui/console-vc.c
index d512f57e10a9..87f57f1c52c6 100644
--- a/ui/console-vc.c
+++ b/ui/console-vc.c
@@ -827,8 +827,7 @@ static void vc_putchar(VCChardev *vc, int ch)
                 case 6:
                     /* report cursor position */
                     response = g_strdup_printf("\033[%d;%dR",
-                           (s->y_base + s->y) % s->total_height + 1,
-                            s->x + 1);
+                                               s->y + 1, s->x + 1);
                     vc_respond_str(vc, response);
                     break;
                 }
-- 
2.43.0

[PATCH v2 4/5] ui/console-vc: add support for cursor DECSC and DECRC commands

[Roman Penyaev]

There are aliases for save and restore cursor commands:

* save cursor
    `ESC 7`     (DEC Save Cursor [1], older VT100)
    `ESC [ s`   (CSI Save Cursor, standard ANSI)

* load cursor
    `ESC 8`     (DEC Restore Cursor [2], older VT100)
    `ESC [ u`   (CSI Restore Cursor, standard ANSI)

This change introduces older DEC sequencies for compatibility with
some scripts (for example [3]) and tools.

This change also adds saving and restoring of character attributes,
which is according to the VT spec [1][2]

[1] https://vt100.net/docs/vt510-rm/DECSC.html
[2] https://vt100.net/docs/vt510-rm/DECRC.html
[3] https://wiki.archlinux.org/title/Working_with_the_serial_console#Resizing_a_terminal

Signed-off-by: Roman Penyaev <r.peniaev@gmail.com>
Cc: "Marc-André Lureau" <marcandre.lureau@redhat.com>
Cc: qemu-devel@nongnu.org
---
 ui/console-vc.c | 40 ++++++++++++++++++++++++++++++++++------
 1 file changed, 34 insertions(+), 6 deletions(-)

diff --git a/ui/console-vc.c b/ui/console-vc.c
index 87f57f1c52c6..522adc2806c8 100644
--- a/ui/console-vc.c
+++ b/ui/console-vc.c
@@ -90,6 +90,7 @@ struct VCChardev {
     int esc_params[MAX_ESC_PARAMS];
     int nb_esc_params;
     TextAttributes t_attrib; /* currently active text attributes */
+    TextAttributes t_attrib_saved;
     int x_saved, y_saved;
 };
 typedef struct VCChardev VCChardev;
@@ -644,6 +645,31 @@ static void vc_set_cursor(VCChardev *vc, int x, int y)
     s->y = y;
 }
 
+/**
+ * vc_save_cursor() - saves cursor position and character attributes.
+ */
+static void vc_save_cursor(VCChardev *vc)
+{
+    QemuTextConsole *s = vc->console;
+
+    vc->x_saved = s->x;
+    vc->y_saved = s->y;
+    vc->t_attrib_saved = vc->t_attrib;
+}
+
+/**
+ * vc_restore_cursor() - restores cursor position and character
+ * attributes from saved state.
+ */
+static void vc_restore_cursor(VCChardev *vc)
+{
+    QemuTextConsole *s = vc->console;
+
+    s->x = vc->x_saved;
+    s->y = vc->y_saved;
+    vc->t_attrib = vc->t_attrib_saved;
+}
+
 static void vc_putchar(VCChardev *vc, int ch)
 {
     QemuTextConsole *s = vc->console;
@@ -699,6 +725,12 @@ static void vc_putchar(VCChardev *vc, int ch)
             vc->state = TTY_STATE_G0;
         } else if (ch == ')') {
             vc->state = TTY_STATE_G1;
+        } else if (ch == '7') {
+            vc_save_cursor(vc);
+            vc->state = TTY_STATE_NORM;
+        } else if (ch == '8') {
+            vc_restore_cursor(vc);
+            vc->state = TTY_STATE_NORM;
         } else {
             vc->state = TTY_STATE_NORM;
         }
@@ -833,14 +865,10 @@ static void vc_putchar(VCChardev *vc, int ch)
                 }
                 break;
             case 's':
-                /* save cursor position */
-                vc->x_saved = s->x;
-                vc->y_saved = s->y;
+                vc_save_cursor(vc);
                 break;
             case 'u':
-                /* restore cursor position */
-                s->x = vc->x_saved;
-                s->y = vc->y_saved;
+                vc_restore_cursor(vc);
                 break;
             default:
                 trace_console_putchar_unhandled(ch);
-- 
2.43.0

[PATCH v2 5/5] ui/console-vc: implement DCH (delete) and ICH (insert) commands

[Roman Penyaev]

This patch implements DCH (delete character) and ICH (insert
character) commands.

DCH - Delete Character:
   "As characters are deleted, the remaining characters between the
    cursor and right margin move to the left. Character attributes move
    with the characters. The terminal adds blank spaces with no visual
    character attributes at the right margin. DCH has no effect outside
    the scrolling margins" [1].

ICH - Insert Character:
   "The ICH sequence inserts Pn blank characters with the normal
    character attribute. The cursor remains at the beginning of the
    blank characters. Text between the cursor and right margin moves to
    the right. Characters scrolled past the right margin are lost. ICH
    has no effect outside the scrolling margins" [2].

Without these commands console is barely usable.

[1] https://vt100.net/docs/vt510-rm/DCH.html
[1] https://vt100.net/docs/vt510-rm/ICH.html

Signed-off-by: Roman Penyaev <r.peniaev@gmail.com>
Cc: "Marc-André Lureau" <marcandre.lureau@redhat.com>
Cc: qemu-devel@nongnu.org
---
 ui/console-vc.c | 88 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 88 insertions(+)

diff --git a/ui/console-vc.c b/ui/console-vc.c
index 522adc2806c8..df1341513d53 100644
--- a/ui/console-vc.c
+++ b/ui/console-vc.c
@@ -645,6 +645,88 @@ static void vc_set_cursor(VCChardev *vc, int x, int y)
     s->y = y;
 }
 
+/**
+ * vc_csi_P() - (DCH) deletes one or more characters from the cursor
+ * position to the right. As characters are deleted, the remaining
+ * characters between the cursor and right margin move to the
+ * left. Character attributes move with the characters.
+ */
+static void vc_csi_P(struct VCChardev *vc, unsigned int nr)
+{
+    QemuTextConsole *s = vc->console;
+    TextCell *c1, *c2;
+    unsigned int x1, x2, y;
+    unsigned int end, len;
+
+    if (!nr) {
+        nr = 1;
+    }
+    if (nr > s->width - s->x) {
+        nr = s->width - s->x;
+        if (!nr) {
+            return;
+        }
+    }
+
+    x1 = s->x;
+    x2 = s->x + nr;
+    len = s->width - x2;
+    if (len) {
+        y = (s->y_base + s->y) % s->total_height;
+        c1 = &s->cells[y * s->width + x1];
+        c2 = &s->cells[y * s->width + x2];
+        memmove(c1, c2, len * sizeof(*c1));
+        for (end = x1 + len; x1 < end; x1++) {
+            vc_update_xy(vc, x1, s->y);
+        }
+    }
+    /* Clear the rest */
+    for (; x1 < s->width; x1++) {
+        vc_clear_xy(vc, x1, s->y);
+    }
+}
+
+/**
+ * vc_csi_at() - (ICH) inserts `nr` blank characters with the default
+ * character attribute. The cursor remains at the beginning of the
+ * blank characters. Text between the cursor and right margin moves to
+ * the right. Characters scrolled past the right margin are lost.
+ */
+static void vc_csi_at(struct VCChardev *vc, unsigned int nr)
+{
+    QemuTextConsole *s = vc->console;
+    TextCell *c1, *c2;
+    unsigned int x1, x2, y;
+    unsigned int end, len;
+
+    if (!nr) {
+        nr = 1;
+    }
+    if (nr > s->width - s->x) {
+        nr = s->width - s->x;
+        if (!nr) {
+            return;
+        }
+    }
+
+    x1 = s->x + nr;
+    x2 = s->x;
+    len = s->width - x1;
+    if (len) {
+        y = (s->y_base + s->y) % s->total_height;
+        c1 = &s->cells[y * s->width + x1];
+        c2 = &s->cells[y * s->width + x2];
+        memmove(c1, c2, len * sizeof(*c1));
+        for (end = x1 + len; x1 < end; x1++) {
+            vc_update_xy(vc, x1, s->y);
+        }
+    }
+    /* Insert blanks */
+    for (x1 = s->x; x1 < s->x + nr; x1++) {
+        vc_clear_xy(vc, x1, s->y);
+    }
+}
+
 /**
  * vc_save_cursor() - saves cursor position and character attributes.
  */
@@ -847,6 +929,9 @@ static void vc_putchar(VCChardev *vc, int ch)
                     break;
                 }
                 break;
+            case 'P':
+                vc_csi_P(vc, vc->esc_params[0]);
+                break;
             case 'm':
                 vc_handle_escape(vc);
                 break;
@@ -870,6 +955,9 @@ static void vc_putchar(VCChardev *vc, int ch)
             case 'u':
                 vc_restore_cursor(vc);
                 break;
+            case '@':
+                vc_csi_at(vc, vc->esc_params[0]);
+                break;
             default:
                 trace_console_putchar_unhandled(ch);
                 break;
-- 
2.43.0

Re: [PATCH v2 0/5] ui/console-vc: various fixes and improvements

[Marc-André Lureau]

Hi,

On Wed, Feb 26, 2025 at 11:59 AM Roman Penyaev <r.peniaev@gmail.com> wrote:
>
> This patch series brings several fixes and improvements to the QEMU
> virtual console (VC), making it more usable and aligning it better
> with VT100 spec. The updates include support for inserting and
> deleting characters, better cursor handling and correct position
> reporting. Without these changes, the console is hardly usable.
>
> 1. Support for DCH (delete) and ICH (insert) commands
>   * Properly handles character deletion and insertion as per VT100
>     specs.
>
> 2. Add support for cursor save/restore (ESC 7 and ESC 8)
>   * Implements legacy DEC sequences for compatibility.
>   * Ensures cursor attributes are also saved and restored.
>
> 3. Fix cursor position reporting
>   * Reports cursor position relative to the screen instead of the
>     scroll buffer.
>   * Fixes issues with tools that rely on accurate cursor placement.
>
> 4. Ensure DSR (Device Status Report) responses go to applications
>   * Stops terminal from rendering responses to the screen but instead
>     send reports back to applications.
>
> 5. Handle `ESC ( <ch>` character set sequences
>   * Makes `top` output look clean by handling the character set
>     commands.
>
> These changes improve the virtual console's compatibility with
> terminal-based apps. Without them, the console is quite frustrating to
> use.
>
> v1 .. v2:
>
> * The VT spec and ECMA-48 standard are somewhat vague about exactly
>   what attributes should be set for the ICH (insert character),
>   but various terminal implementations reset the character to the
>   default state. Instead of inserting a "space" with the current
>   attribute, call `vc_clear_xy()` for the inserted character.
>
> Signed-off-by: Roman Penyaev <r.peniaev@gmail.com>
> Cc: "Marc-André Lureau" <marcandre.lureau@redhat.com>
> Cc: Kevin Wolf <kwolf@redhat.com>
> Cc: Daniel P. Berrange <berrange@redhat.com>
> Cc: qemu-devel@nongnu.org

lgtm,

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>

>
> Roman Penyaev (5):
>   ui/console-vc: introduce parsing of the 'ESC ( <ch>' sequence
>   ui/console-vc: report to the application instead of screen rendering
>   ui/console-vc: report cursor position in the screen not in the scroll
>     buffer
>   ui/console-vc: add support for cursor DECSC and DECRC commands
>   ui/console-vc: implement DCH (delete) and ICH (insert) commands
>
>  ui/console-vc.c | 154 ++++++++++++++++++++++++++++++++++++++++++++----
>  1 file changed, 142 insertions(+), 12 deletions(-)
>
> --
> 2.43.0
>

Re: [PATCH v2 0/5] ui/console-vc: various fixes and improvements

[Roman Penyaev]

Hi Marc-André,

Do you want me to resend the series with the review
tag incorporated? Or are you fine to pull as-is if there
is nothing left to address from my side?

--
Roman



On Wed, Feb 26, 2025 at 11:06 AM Marc-André Lureau
<marcandre.lureau@redhat.com> wrote:
>
> Hi,
>
> On Wed, Feb 26, 2025 at 11:59 AM Roman Penyaev <r.peniaev@gmail.com> wrote:
> >
> > This patch series brings several fixes and improvements to the QEMU
> > virtual console (VC), making it more usable and aligning it better
> > with VT100 spec. The updates include support for inserting and
> > deleting characters, better cursor handling and correct position
> > reporting. Without these changes, the console is hardly usable.
> >
> > 1. Support for DCH (delete) and ICH (insert) commands
> >   * Properly handles character deletion and insertion as per VT100
> >     specs.
> >
> > 2. Add support for cursor save/restore (ESC 7 and ESC 8)
> >   * Implements legacy DEC sequences for compatibility.
> >   * Ensures cursor attributes are also saved and restored.
> >
> > 3. Fix cursor position reporting
> >   * Reports cursor position relative to the screen instead of the
> >     scroll buffer.
> >   * Fixes issues with tools that rely on accurate cursor placement.
> >
> > 4. Ensure DSR (Device Status Report) responses go to applications
> >   * Stops terminal from rendering responses to the screen but instead
> >     send reports back to applications.
> >
> > 5. Handle `ESC ( <ch>` character set sequences
> >   * Makes `top` output look clean by handling the character set
> >     commands.
> >
> > These changes improve the virtual console's compatibility with
> > terminal-based apps. Without them, the console is quite frustrating to
> > use.
> >
> > v1 .. v2:
> >
> > * The VT spec and ECMA-48 standard are somewhat vague about exactly
> >   what attributes should be set for the ICH (insert character),
> >   but various terminal implementations reset the character to the
> >   default state. Instead of inserting a "space" with the current
> >   attribute, call `vc_clear_xy()` for the inserted character.
> >
> > Signed-off-by: Roman Penyaev <r.peniaev@gmail.com>
> > Cc: "Marc-André Lureau" <marcandre.lureau@redhat.com>
> > Cc: Kevin Wolf <kwolf@redhat.com>
> > Cc: Daniel P. Berrange <berrange@redhat.com>
> > Cc: qemu-devel@nongnu.org
>
> lgtm,
>
> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>
> >
> > Roman Penyaev (5):
> >   ui/console-vc: introduce parsing of the 'ESC ( <ch>' sequence
> >   ui/console-vc: report to the application instead of screen rendering
> >   ui/console-vc: report cursor position in the screen not in the scroll
> >     buffer
> >   ui/console-vc: add support for cursor DECSC and DECRC commands
> >   ui/console-vc: implement DCH (delete) and ICH (insert) commands
> >
> >  ui/console-vc.c | 154 ++++++++++++++++++++++++++++++++++++++++++++----
> >  1 file changed, 142 insertions(+), 12 deletions(-)
> >
> > --
> > 2.43.0
> >
>