Git development
 help / color / mirror / Atom feed
* [PATCH] utf8: make display_mode_esc_sequence_len() take a length
@ 2026-11-17 16:50 Junio C Hamano
  2026-11-17 20:06 ` [PATCH v2] utf8: add display_mode_esc_sequence_len_counted() Junio C Hamano
  0 siblings, 1 reply; 11+ messages in thread
From: Junio C Hamano @ 2026-11-17 16:50 UTC (permalink / raw)
  To: git; +Cc: Dirk Gouders

display_mode_esc_sequence_len() scans for an ANSI display mode
escape sequence (e.g. "ESC [ 1 ; 31 m"), relying on NUL termination
to stop.  Some callers, however, use a counted <ptr, len> buffer or
need to stop at a specific position.

Teach it to take a buffer length and never read at or beyond
ptr[len], returning the length of a complete sequence at the start
or 0 otherwise.

Update the callers to pass the remaining byte count.  For
strbuf_add_wrapped_text(), which takes a NUL-terminated string,
compute the end once upfront instead of calling strlen() each time.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 * Copy of https://lore.kernel.org/git/xmqqse0zfn2n.fsf@gitster.g/
   with a proposed commit log message.

 pretty.c |  3 ++-
 utf8.c   | 27 ++++++++++++++++++---------
 utf8.h   |  2 +-
 3 files changed, 21 insertions(+), 11 deletions(-)

diff --git a/pretty.c b/pretty.c
index 0bc8ad8a9a..a20fd61909 100644
--- a/pretty.c
+++ b/pretty.c
@@ -1834,7 +1834,8 @@ static size_t format_and_pad_commit(struct strbuf *sb, /* in UTF-8 */
 			while (p > sb->buf && ch - p < 10 && *p != '\033')
 				p--;
 			if (*p != '\033' ||
-			    ch + 1 - p != display_mode_esc_sequence_len(p))
+			    ch + 1 - p != display_mode_esc_sequence_len(p,
+								ch + 1 - p))
 				break;
 			/*
 			 * got a good ansi sequence, put it back to
diff --git a/utf8.c b/utf8.c
index 96460cc414..544c5329f3 100644
--- a/utf8.c
+++ b/utf8.c
@@ -16,16 +16,23 @@ struct interval {
 	ucs_char_t last;
 };
 
-size_t display_mode_esc_sequence_len(const char *s)
+/*
+ * Return the length of the ANSI display mode escape sequence (e.g.,
+ * "\033[1;31m") that appears at the beginning of the "len" bytes
+ * of string "s", or 0 if the bytes do not begin with a complete
+ * such sequence.  Never looks at s[len] or beyond.
+ */
+size_t display_mode_esc_sequence_len(const char *s, size_t len)
 {
-	const char *p = s;
-	if (*p++ != '\033')
+	const char *p = s, *end = s + len;
+
+	if (end <= p || *p++ != '\033')
 		return 0;
-	if (*p++ != '[')
+	if (end <= p || *p++ != '[')
 		return 0;
-	while (isdigit(*p) || *p == ';')
+	while (p < end && (isdigit(*p) || *p == ';'))
 		p++;
-	if (*p++ != 'm')
+	if (end <= p || *p++ != 'm')
 		return 0;
 	return p - s;
 }
@@ -218,7 +225,8 @@ int utf8_strnwidth(const char *string, size_t len, int skip_ansi)
 		size_t skip;
 
 		while (skip_ansi &&
-		       (skip = display_mode_esc_sequence_len(string)) != 0)
+		       (skip = display_mode_esc_sequence_len(string,
+							     orig + len - string)) != 0)
 			string += skip;
 
 		glyph_width = utf8_width(&string, NULL);
@@ -279,6 +287,7 @@ void strbuf_add_wrapped_text(struct strbuf *buf,
 {
 	int indent, w, assume_utf8 = 1;
 	const char *bol, *space, *start = text;
+	const char *end = text + strlen(text);
 	size_t orig_len = buf->len;
 
 	if (width <= 0) {
@@ -299,7 +308,7 @@ void strbuf_add_wrapped_text(struct strbuf *buf,
 		char c;
 		size_t skip;
 
-		while ((skip = display_mode_esc_sequence_len(text)))
+		while ((skip = display_mode_esc_sequence_len(text, end - text)))
 			text += skip;
 
 		c = *text;
@@ -378,7 +387,7 @@ void strbuf_utf8_replace(struct strbuf *sb_src, int pos, int width,
 		int glyph_width;
 		size_t n;
 
-		while ((n = display_mode_esc_sequence_len(src))) {
+		while ((n = display_mode_esc_sequence_len(src, end - src))) {
 			strbuf_add(&dst, src, n);
 			src += n;
 		}
diff --git a/utf8.h b/utf8.h
index cf8ecb0f21..fb2227a9e2 100644
--- a/utf8.h
+++ b/utf8.h
@@ -5,7 +5,7 @@ struct strbuf;
 
 typedef unsigned int ucs_char_t;  /* assuming 32bit int */
 
-size_t display_mode_esc_sequence_len(const char *s);
+size_t display_mode_esc_sequence_len(const char *s, size_t len);
 int utf8_width(const char **start, size_t *remainder_p);
 int utf8_strnwidth(const char *string, size_t len, int skip_ansi);
 int utf8_strwidth(const char *string);
-- 
2.56.1-740-gf38658f3d0


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH v2] utf8: add display_mode_esc_sequence_len_counted()
  2026-11-17 16:50 [PATCH] utf8: make display_mode_esc_sequence_len() take a length Junio C Hamano
@ 2026-11-17 20:06 ` Junio C Hamano
  2026-11-17 21:26   ` Dirk Gouders
  2026-11-17 23:08   ` Andy Koppe
  0 siblings, 2 replies; 11+ messages in thread
From: Junio C Hamano @ 2026-11-17 20:06 UTC (permalink / raw)
  To: git; +Cc: Dirk Gouders

display_mode_esc_sequence_len() scans for an ANSI display mode
escape sequence (e.g. "ESC [ 1 ; 31 m") at the start of a string,
relying on NUL termination to stop.  Some callers, however, work on
a counted buffer: utf8_strnwidth() takes <string, len> and must not
scan past it, while strbuf_utf8_replace() and
strbuf_add_wrapped_text() know (or can cheaply learn) the end.

Add display_mode_esc_sequence_len_counted() taking <ptr, len>,
which never reads at or beyond ptr[len], and make
display_mode_esc_sequence_len() a thin wrapper for NUL-terminated
strings.

Use the counted variant in utf8_strnwidth() for correctness, and in
strbuf_utf8_replace() and strbuf_add_wrapped_text() (computing the
end once upfront in the latter) to avoid measuring the remaining
text in each iteration.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 * As there are new patches that want to add new callers to this
   function, let's keep the original with the same API while adding
   a new helper function that allows us to limit the scan.

 utf8.c | 35 ++++++++++++++++++++++++++---------
 utf8.h |  1 +
 2 files changed, 27 insertions(+), 9 deletions(-)

diff --git a/utf8.c b/utf8.c
index 96460cc414..d0546ab31f 100644
--- a/utf8.c
+++ b/utf8.c
@@ -16,20 +16,33 @@ struct interval {
 	ucs_char_t last;
 };
 
-size_t display_mode_esc_sequence_len(const char *s)
+/*
+ * Return the length of the ANSI display mode escape sequence (e.g.,
+ * "\033[1;31m") that appears at the beginning of the "len" bytes
+ * of string "s", or 0 if the bytes do not begin with a complete
+ * such sequence.  Never looks at s[len] or beyond.
+ */
+size_t display_mode_esc_sequence_len_counted(const char *s, size_t len)
 {
-	const char *p = s;
-	if (*p++ != '\033')
+	const char *p = s, *end = s + len;
+
+	if (end <= p || *p++ != '\033')
 		return 0;
-	if (*p++ != '[')
+	if (end <= p || *p++ != '[')
 		return 0;
-	while (isdigit(*p) || *p == ';')
+	while (p < end && (isdigit(*p) || *p == ';'))
 		p++;
-	if (*p++ != 'm')
+	if (end <= p || *p++ != 'm')
 		return 0;
 	return p - s;
 }
 
+/* Same as above, but for a NUL-terminated string "s". */
+size_t display_mode_esc_sequence_len(const char *s)
+{
+	return display_mode_esc_sequence_len_counted(s, strlen(s));
+}
+
 /* auxiliary function for binary search in interval table */
 static int bisearch(ucs_char_t ucs, const struct interval *table, int max)
 {
@@ -218,7 +231,8 @@ int utf8_strnwidth(const char *string, size_t len, int skip_ansi)
 		size_t skip;
 
 		while (skip_ansi &&
-		       (skip = display_mode_esc_sequence_len(string)) != 0)
+		       (skip = display_mode_esc_sequence_len_counted(string,
+							orig + len - string)) != 0)
 			string += skip;
 
 		glyph_width = utf8_width(&string, NULL);
@@ -279,6 +293,7 @@ void strbuf_add_wrapped_text(struct strbuf *buf,
 {
 	int indent, w, assume_utf8 = 1;
 	const char *bol, *space, *start = text;
+	const char *end = text + strlen(text);
 	size_t orig_len = buf->len;
 
 	if (width <= 0) {
@@ -299,7 +314,8 @@ void strbuf_add_wrapped_text(struct strbuf *buf,
 		char c;
 		size_t skip;
 
-		while ((skip = display_mode_esc_sequence_len(text)))
+		while ((skip = display_mode_esc_sequence_len_counted(text,
+								end - text)))
 			text += skip;
 
 		c = *text;
@@ -378,7 +394,8 @@ void strbuf_utf8_replace(struct strbuf *sb_src, int pos, int width,
 		int glyph_width;
 		size_t n;
 
-		while ((n = display_mode_esc_sequence_len(src))) {
+		while ((n = display_mode_esc_sequence_len_counted(src,
+								  end - src))) {
 			strbuf_add(&dst, src, n);
 			src += n;
 		}
diff --git a/utf8.h b/utf8.h
index cf8ecb0f21..57ca2ab270 100644
--- a/utf8.h
+++ b/utf8.h
@@ -6,6 +6,7 @@ struct strbuf;
 typedef unsigned int ucs_char_t;  /* assuming 32bit int */
 
 size_t display_mode_esc_sequence_len(const char *s);
+size_t display_mode_esc_sequence_len_counted(const char *s, size_t len);
 int utf8_width(const char **start, size_t *remainder_p);
 int utf8_strnwidth(const char *string, size_t len, int skip_ansi);
 int utf8_strwidth(const char *string);
-- 
2.56.1-740-gf38658f3d0


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted()
  2026-11-17 20:06 ` [PATCH v2] utf8: add display_mode_esc_sequence_len_counted() Junio C Hamano
@ 2026-11-17 21:26   ` Dirk Gouders
  2026-11-17 21:43     ` Junio C Hamano
  2026-11-17 23:08   ` Andy Koppe
  1 sibling, 1 reply; 11+ messages in thread
From: Dirk Gouders @ 2026-11-17 21:26 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Tue, Nov 17 2026, Junio C Hamano wrote:

> display_mode_esc_sequence_len() scans for an ANSI display mode
> escape sequence (e.g. "ESC [ 1 ; 31 m") at the start of a string,
> relying on NUL termination to stop.  Some callers, however, work on
> a counted buffer: utf8_strnwidth() takes <string, len> and must not
> scan past it, while strbuf_utf8_replace() and
> strbuf_add_wrapped_text() know (or can cheaply learn) the end.
>
> Add display_mode_esc_sequence_len_counted() taking <ptr, len>,
> which never reads at or beyond ptr[len], and make
> display_mode_esc_sequence_len() a thin wrapper for NUL-terminated
> strings.
>
> Use the counted variant in utf8_strnwidth() for correctness, and in
> strbuf_utf8_replace() and strbuf_add_wrapped_text() (computing the
> end once upfront in the latter) to avoid measuring the remaining
> text in each iteration.
>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>  * As there are new patches that want to add new callers to this
>    function, let's keep the original with the same API while adding
>    a new helper function that allows us to limit the scan.
>
>  utf8.c | 35 ++++++++++++++++++++++++++---------
>  utf8.h |  1 +
>  2 files changed, 27 insertions(+), 9 deletions(-)
>
> diff --git a/utf8.c b/utf8.c
> index 96460cc414..d0546ab31f 100644
> --- a/utf8.c
> +++ b/utf8.c
> @@ -16,20 +16,33 @@ struct interval {
>  	ucs_char_t last;
>  };
>  
> -size_t display_mode_esc_sequence_len(const char *s)
> +/*
> + * Return the length of the ANSI display mode escape sequence (e.g.,
> + * "\033[1;31m") that appears at the beginning of the "len" bytes
> + * of string "s", or 0 if the bytes do not begin with a complete
> + * such sequence.  Never looks at s[len] or beyond.
> + */
> +size_t display_mode_esc_sequence_len_counted(const char *s, size_t len)
>  {
> -	const char *p = s;
> -	if (*p++ != '\033')
> +	const char *p = s, *end = s + len;

You have better knowledge of the users of this function: would an early
return perhaps handle a significant amount of cases?

        /* The shortest SGR sequence is "\033[m". */
        if (len < 3)
                return 0;


And, why not add some tests (the untested function I showed had
an off-by-one error):

diff --git a/Makefile b/Makefile
index 4c84eea6e2..3c8dc9ca40 100644
--- a/Makefile
+++ b/Makefile
@@ -1564,6 +1564,7 @@ CLAR_TEST_SUITES += u-strvec
 CLAR_TEST_SUITES += u-trailer
 CLAR_TEST_SUITES += u-urlmatch-normalization
 CLAR_TEST_SUITES += u-utf8-width
+CLAR_TEST_SUITES += u-utf8-ansi
 CLAR_TEST_PROG = $(UNIT_TEST_BIN)/unit-tests$(X)
 CLAR_TEST_OBJS = $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(CLAR_TEST_SUITES))
 CLAR_TEST_OBJS += $(UNIT_TEST_DIR)/clar/clar.o
diff --git a/t/meson.build b/t/meson.build
index 1cf1d95f70..e224df58c8 100644
--- a/t/meson.build
+++ b/t/meson.build
@@ -27,6 +27,7 @@ clar_test_suites = [
   'unit-tests/u-trailer.c',
   'unit-tests/u-urlmatch-normalization.c',
   'unit-tests/u-utf8-width.c',
+  'unit-tests/u-utf8-ansi.c',
 ]
 
 clar_sources = [
diff --git a/t/unit-tests/u-utf8-ansi.c b/t/unit-tests/u-utf8-ansi.c
new file mode 100644
index 0000000000..c19d8bce78
--- /dev/null
+++ b/t/unit-tests/u-utf8-ansi.c
@@ -0,0 +1,14 @@
+#include "unit-test.h"
+#include "utf8.h"
+#include "strbuf.h"
+
+/*
+ * Test len limited functions for ANSI SGR lengths.
+ */
+void test_utf8_ansi__sgr_len(void)
+{
+	cl_assert_equal_i(0, display_mode_esc_sequence_len_counted("\033[m", 2));
+	cl_assert_equal_i(3, display_mode_esc_sequence_len_counted("\033[m", 3));
+	cl_assert_equal_i(5, display_mode_esc_sequence_len_counted("\033[31m", 5));
+	cl_assert_equal_i(7, display_mode_esc_sequence_len_counted("\033[0;31m", 7));
+}

Regards,

Dirk

> +	if (end <= p || *p++ != '\033')
>  		return 0;
> -	if (*p++ != '[')
> +	if (end <= p || *p++ != '[')
>  		return 0;
> -	while (isdigit(*p) || *p == ';')
> +	while (p < end && (isdigit(*p) || *p == ';'))
>  		p++;
> -	if (*p++ != 'm')
> +	if (end <= p || *p++ != 'm')
>  		return 0;
>  	return p - s;
>  }
>  
> +/* Same as above, but for a NUL-terminated string "s". */
> +size_t display_mode_esc_sequence_len(const char *s)
> +{
> +	return display_mode_esc_sequence_len_counted(s, strlen(s));
> +}
> +
>  /* auxiliary function for binary search in interval table */
>  static int bisearch(ucs_char_t ucs, const struct interval *table, int max)
>  {
> @@ -218,7 +231,8 @@ int utf8_strnwidth(const char *string, size_t len, int skip_ansi)
>  		size_t skip;
>  
>  		while (skip_ansi &&
> -		       (skip = display_mode_esc_sequence_len(string)) != 0)
> +		       (skip = display_mode_esc_sequence_len_counted(string,
> +							orig + len - string)) != 0)
>  			string += skip;
>  
>  		glyph_width = utf8_width(&string, NULL);
> @@ -279,6 +293,7 @@ void strbuf_add_wrapped_text(struct strbuf *buf,
>  {
>  	int indent, w, assume_utf8 = 1;
>  	const char *bol, *space, *start = text;
> +	const char *end = text + strlen(text);
>  	size_t orig_len = buf->len;
>  
>  	if (width <= 0) {
> @@ -299,7 +314,8 @@ void strbuf_add_wrapped_text(struct strbuf *buf,
>  		char c;
>  		size_t skip;
>  
> -		while ((skip = display_mode_esc_sequence_len(text)))
> +		while ((skip = display_mode_esc_sequence_len_counted(text,
> +								end - text)))
>  			text += skip;
>  
>  		c = *text;
> @@ -378,7 +394,8 @@ void strbuf_utf8_replace(struct strbuf *sb_src, int pos, int width,
>  		int glyph_width;
>  		size_t n;
>  
> -		while ((n = display_mode_esc_sequence_len(src))) {
> +		while ((n = display_mode_esc_sequence_len_counted(src,
> +								  end - src))) {
>  			strbuf_add(&dst, src, n);
>  			src += n;
>  		}
> diff --git a/utf8.h b/utf8.h
> index cf8ecb0f21..57ca2ab270 100644
> --- a/utf8.h
> +++ b/utf8.h
> @@ -6,6 +6,7 @@ struct strbuf;
>  typedef unsigned int ucs_char_t;  /* assuming 32bit int */
>  
>  size_t display_mode_esc_sequence_len(const char *s);
> +size_t display_mode_esc_sequence_len_counted(const char *s, size_t len);
>  int utf8_width(const char **start, size_t *remainder_p);
>  int utf8_strnwidth(const char *string, size_t len, int skip_ansi);
>  int utf8_strwidth(const char *string);

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted()
  2026-11-17 21:26   ` Dirk Gouders
@ 2026-11-17 21:43     ` Junio C Hamano
  2026-11-17 22:03       ` Dirk Gouders
  0 siblings, 1 reply; 11+ messages in thread
From: Junio C Hamano @ 2026-11-17 21:43 UTC (permalink / raw)
  To: Dirk Gouders; +Cc: git

Dirk Gouders <dirk@gouders.net> writes:

> You have better knowledge of the users of this function: would an early
> return perhaps handle a significant amount of cases?
>
>         /* The shortest SGR sequence is "\033[m". */
>         if (len < 3)
>                 return 0;

Do you have any benchmarks?  I don't.

I find it of dubious performance merit to run this comparison for a
long string as it is eaten one UTF-8 characters at a time by the
caller, and when the length is actually shorter than 3, then
runnning the remainder of the function for a few steps should not be
10x more expensive than the above comparison, no?

> And, why not add some tests (the untested function I showed had
> an off-by-one error):

Sounds good.  Should I just steal what you sent out?

> diff --git a/Makefile b/Makefile
> index 4c84eea6e2..3c8dc9ca40 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1564,6 +1564,7 @@ CLAR_TEST_SUITES += u-strvec
>  CLAR_TEST_SUITES += u-trailer
>  CLAR_TEST_SUITES += u-urlmatch-normalization
>  CLAR_TEST_SUITES += u-utf8-width
> +CLAR_TEST_SUITES += u-utf8-ansi
>  CLAR_TEST_PROG = $(UNIT_TEST_BIN)/unit-tests$(X)
>  CLAR_TEST_OBJS = $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(CLAR_TEST_SUITES))
>  CLAR_TEST_OBJS += $(UNIT_TEST_DIR)/clar/clar.o
> diff --git a/t/meson.build b/t/meson.build
> index 1cf1d95f70..e224df58c8 100644
> --- a/t/meson.build
> +++ b/t/meson.build
> @@ -27,6 +27,7 @@ clar_test_suites = [
>    'unit-tests/u-trailer.c',
>    'unit-tests/u-urlmatch-normalization.c',
>    'unit-tests/u-utf8-width.c',
> +  'unit-tests/u-utf8-ansi.c',
>  ]
>  
>  clar_sources = [
> diff --git a/t/unit-tests/u-utf8-ansi.c b/t/unit-tests/u-utf8-ansi.c
> new file mode 100644
> index 0000000000..c19d8bce78
> --- /dev/null
> +++ b/t/unit-tests/u-utf8-ansi.c
> @@ -0,0 +1,14 @@
> +#include "unit-test.h"
> +#include "utf8.h"
> +#include "strbuf.h"
> +
> +/*
> + * Test len limited functions for ANSI SGR lengths.
> + */
> +void test_utf8_ansi__sgr_len(void)
> +{
> +	cl_assert_equal_i(0, display_mode_esc_sequence_len_counted("\033[m", 2));
> +	cl_assert_equal_i(3, display_mode_esc_sequence_len_counted("\033[m", 3));
> +	cl_assert_equal_i(5, display_mode_esc_sequence_len_counted("\033[31m", 5));
> +	cl_assert_equal_i(7, display_mode_esc_sequence_len_counted("\033[0;31m", 7));
> +}

Thanks.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted()
  2026-11-17 21:43     ` Junio C Hamano
@ 2026-11-17 22:03       ` Dirk Gouders
  2026-11-18  1:33         ` Junio C Hamano
  0 siblings, 1 reply; 11+ messages in thread
From: Dirk Gouders @ 2026-11-17 22:03 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Tue, Nov 17 2026, Junio C Hamano wrote:

> Dirk Gouders <dirk@gouders.net> writes:
>
>> You have better knowledge of the users of this function: would an early
>> return perhaps handle a significant amount of cases?
>>
>>         /* The shortest SGR sequence is "\033[m". */
>>         if (len < 3)
>>                 return 0;
>
> Do you have any benchmarks?  I don't.
>
> I find it of dubious performance merit to run this comparison for a
> long string as it is eaten one UTF-8 characters at a time by the
> caller, and when the length is actually shorter than 3, then
> runnning the remainder of the function for a few steps should not be
> 10x more expensive than the above comparison, no?

Meanwhile, I thought: even if the test would be a good idea, the
"\033"-test probably is the one that handles most cases.

Maybe I will look closer, but not now.  Correctness is more important
and I need to work on v2.

>> And, why not add some tests (the untested function I showed had
>> an off-by-one error):
>
> Sounds good.  Should I just steal what you sent out?

That wouldn't be stealing, of course you can use it.

Regards,

Dirk

>> diff --git a/Makefile b/Makefile
>> index 4c84eea6e2..3c8dc9ca40 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -1564,6 +1564,7 @@ CLAR_TEST_SUITES += u-strvec
>>  CLAR_TEST_SUITES += u-trailer
>>  CLAR_TEST_SUITES += u-urlmatch-normalization
>>  CLAR_TEST_SUITES += u-utf8-width
>> +CLAR_TEST_SUITES += u-utf8-ansi
>>  CLAR_TEST_PROG = $(UNIT_TEST_BIN)/unit-tests$(X)
>>  CLAR_TEST_OBJS = $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(CLAR_TEST_SUITES))
>>  CLAR_TEST_OBJS += $(UNIT_TEST_DIR)/clar/clar.o
>> diff --git a/t/meson.build b/t/meson.build
>> index 1cf1d95f70..e224df58c8 100644
>> --- a/t/meson.build
>> +++ b/t/meson.build
>> @@ -27,6 +27,7 @@ clar_test_suites = [
>>    'unit-tests/u-trailer.c',
>>    'unit-tests/u-urlmatch-normalization.c',
>>    'unit-tests/u-utf8-width.c',
>> +  'unit-tests/u-utf8-ansi.c',
>>  ]
>>  
>>  clar_sources = [
>> diff --git a/t/unit-tests/u-utf8-ansi.c b/t/unit-tests/u-utf8-ansi.c
>> new file mode 100644
>> index 0000000000..c19d8bce78
>> --- /dev/null
>> +++ b/t/unit-tests/u-utf8-ansi.c
>> @@ -0,0 +1,14 @@
>> +#include "unit-test.h"
>> +#include "utf8.h"
>> +#include "strbuf.h"
>> +
>> +/*
>> + * Test len limited functions for ANSI SGR lengths.
>> + */
>> +void test_utf8_ansi__sgr_len(void)
>> +{
>> +	cl_assert_equal_i(0, display_mode_esc_sequence_len_counted("\033[m", 2));
>> +	cl_assert_equal_i(3, display_mode_esc_sequence_len_counted("\033[m", 3));
>> +	cl_assert_equal_i(5, display_mode_esc_sequence_len_counted("\033[31m", 5));
>> +	cl_assert_equal_i(7, display_mode_esc_sequence_len_counted("\033[0;31m", 7));
>> +}
>
> Thanks.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted()
  2026-11-17 20:06 ` [PATCH v2] utf8: add display_mode_esc_sequence_len_counted() Junio C Hamano
  2026-11-17 21:26   ` Dirk Gouders
@ 2026-11-17 23:08   ` Andy Koppe
  2026-11-17 23:25     ` Junio C Hamano
  1 sibling, 1 reply; 11+ messages in thread
From: Andy Koppe @ 2026-11-17 23:08 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Dirk Gouders

Junio C Hamano wrote:

> -       while (isdigit(*p) || *p == ';')
> +       while (p < end && (isdigit(*p) || *p == ';'))

While you're there, could you add colons (':') to that?

ECMA-48 §5.4.2 defines them as sub-separators within the
semicolon-separated parameters of control sequences. Colons are
referred to as "3/10" there, aka ASCII 0x3A:

https://ecma-international.org/wp-content/uploads/ECMA-48_5th_edition_june_1991.pdf#page=26

Maybe more practically relevant, xterm and its many
more-or-less-compatible brethren accept colons in SGR sequences:

"xterm allows either colons (standard) or semicolons (legacy) to
separate the subparameters (but after the first colon, colons must be
used)"
https://invisible-island.net/xterm/ctlseqs/ctlseqs.html

Andy

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted()
  2026-11-17 23:08   ` Andy Koppe
@ 2026-11-17 23:25     ` Junio C Hamano
  2026-11-18  6:59       ` Andy Koppe
  0 siblings, 1 reply; 11+ messages in thread
From: Junio C Hamano @ 2026-11-17 23:25 UTC (permalink / raw)
  To: Andy Koppe; +Cc: git, Dirk Gouders

Andy Koppe <andy.koppe@gmail.com> writes:

> Junio C Hamano wrote:
>
>> -       while (isdigit(*p) || *p == ';')
>> +       while (p < end && (isdigit(*p) || *p == ';'))
>
> While you're there, could you add colons (':') to that?

Do we generate such a sequence that uses subseparators?

IIRC, this code is not counting the display width of end-user
supplied string.  Rather, it gets string we paint in a buffer and
then counts how much display column it occupies.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted()
  2026-11-17 22:03       ` Dirk Gouders
@ 2026-11-18  1:33         ` Junio C Hamano
  2026-11-18  7:37           ` Dirk Gouders
  0 siblings, 1 reply; 11+ messages in thread
From: Junio C Hamano @ 2026-11-18  1:33 UTC (permalink / raw)
  To: Dirk Gouders; +Cc: git

Dirk Gouders <dirk@gouders.net> writes:

> Meanwhile, I thought: even if the test would be a good idea, the
> "\033"-test probably is the one that handles most cases.
>
> Maybe I will look closer, but not now.  Correctness is more important
> and I need to work on v2.

It would be nice if you can take this patch over and make it your
[PATCH v2 1/2].

Thanks.


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted()
  2026-11-17 23:25     ` Junio C Hamano
@ 2026-11-18  6:59       ` Andy Koppe
  2026-11-18  7:43         ` Dirk Gouders
  0 siblings, 1 reply; 11+ messages in thread
From: Andy Koppe @ 2026-11-18  6:59 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Dirk Gouders

Junio C Hamano wrote:
>
> Andy Koppe <andy.koppe@gmail.com> writes:
>
> > Junio C Hamano wrote:
> >
> >> -       while (isdigit(*p) || *p == ';')
> >> +       while (p < end && (isdigit(*p) || *p == ';'))
> >
> > While you're there, could you add colons (':') to that?
>
> Do we generate such a sequence that uses subseparators?
>
> IIRC, this code is not counting the display width of end-user
> supplied string.  Rather, it gets string we paint in a buffer and
> then counts how much display column it occupies.

You're right that Git itself doesn't generate them (although future
enhancements might change that).

However, display_mode_esc_sequence_len() already operates on expanded
custom formats and message text, e.g. in rewrap_message_tail() via
strbuf_wrap(). And on stdin in git-column. They could all contain SGR
sequences with colons, whose width currently isn't ignored as
intended. (And my proposed make_line_colors_self_contained() wouldn't
reset or replay them.)

The check could be changed to `*p >= '0' && *p<=';'` to cheaply include colons.

Andy

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted()
  2026-11-18  1:33         ` Junio C Hamano
@ 2026-11-18  7:37           ` Dirk Gouders
  0 siblings, 0 replies; 11+ messages in thread
From: Dirk Gouders @ 2026-11-18  7:37 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Tue, Nov 17 2026, Junio C Hamano wrote:

> Dirk Gouders <dirk@gouders.net> writes:
>
>> Meanwhile, I thought: even if the test would be a good idea, the
>> "\033"-test probably is the one that handles most cases.
>>
>> Maybe I will look closer, but not now.  Correctness is more important
>> and I need to work on v2.
>
> It would be nice if you can take this patch over and make it your
> [PATCH v2 1/2].

OK, I will do so, thank you.

Regards,

Dirk

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted()
  2026-11-18  6:59       ` Andy Koppe
@ 2026-11-18  7:43         ` Dirk Gouders
  0 siblings, 0 replies; 11+ messages in thread
From: Dirk Gouders @ 2026-11-18  7:43 UTC (permalink / raw)
  To: Andy Koppe; +Cc: Junio C Hamano, git

On Wed, Nov 18 2026, Andy Koppe wrote:

> Junio C Hamano wrote:
>>
>> Andy Koppe <andy.koppe@gmail.com> writes:
>>
>> > Junio C Hamano wrote:
>> >
>> >> -       while (isdigit(*p) || *p == ';')
>> >> +       while (p < end && (isdigit(*p) || *p == ';'))
>> >
>> > While you're there, could you add colons (':') to that?
>>
>> Do we generate such a sequence that uses subseparators?
>>
>> IIRC, this code is not counting the display width of end-user
>> supplied string.  Rather, it gets string we paint in a buffer and
>> then counts how much display column it occupies.
>
> You're right that Git itself doesn't generate them (although future
> enhancements might change that).
>
> However, display_mode_esc_sequence_len() already operates on expanded
> custom formats and message text, e.g. in rewrap_message_tail() via
> strbuf_wrap(). And on stdin in git-column. They could all contain SGR
> sequences with colons, whose width currently isn't ignored as
> intended. (And my proposed make_line_colors_self_contained() wouldn't
> reset or replay them.)
>
> The check could be changed to `*p >= '0' && *p<=';'` to cheaply include colons.

I would suggest to not mix these changes simply for bisection purposes.

And as far as I know, we would need an FSM to identify valid sequences
if we are going to allow colons.

Regards,

Dirk

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2026-11-18  7:43 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-17 16:50 [PATCH] utf8: make display_mode_esc_sequence_len() take a length Junio C Hamano
2026-11-17 20:06 ` [PATCH v2] utf8: add display_mode_esc_sequence_len_counted() Junio C Hamano
2026-11-17 21:26   ` Dirk Gouders
2026-11-17 21:43     ` Junio C Hamano
2026-11-17 22:03       ` Dirk Gouders
2026-11-18  1:33         ` Junio C Hamano
2026-11-18  7:37           ` Dirk Gouders
2026-11-17 23:08   ` Andy Koppe
2026-11-17 23:25     ` Junio C Hamano
2026-11-18  6:59       ` Andy Koppe
2026-11-18  7:43         ` Dirk Gouders

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox