* [PATCH] utf8: make display_mode_esc_sequence_len() take a length
@ 2026-11-17 16:50 Junio C Hamano
2026-11-17 20:06 ` [PATCH v2] utf8: add display_mode_esc_sequence_len_counted() Junio C Hamano
0 siblings, 1 reply; 11+ messages in thread
From: Junio C Hamano @ 2026-11-17 16:50 UTC (permalink / raw)
To: git; +Cc: Dirk Gouders
display_mode_esc_sequence_len() scans for an ANSI display mode
escape sequence (e.g. "ESC [ 1 ; 31 m"), relying on NUL termination
to stop. Some callers, however, use a counted <ptr, len> buffer or
need to stop at a specific position.
Teach it to take a buffer length and never read at or beyond
ptr[len], returning the length of a complete sequence at the start
or 0 otherwise.
Update the callers to pass the remaining byte count. For
strbuf_add_wrapped_text(), which takes a NUL-terminated string,
compute the end once upfront instead of calling strlen() each time.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
* Copy of https://lore.kernel.org/git/xmqqse0zfn2n.fsf@gitster.g/
with a proposed commit log message.
pretty.c | 3 ++-
utf8.c | 27 ++++++++++++++++++---------
utf8.h | 2 +-
3 files changed, 21 insertions(+), 11 deletions(-)
diff --git a/pretty.c b/pretty.c
index 0bc8ad8a9a..a20fd61909 100644
--- a/pretty.c
+++ b/pretty.c
@@ -1834,7 +1834,8 @@ static size_t format_and_pad_commit(struct strbuf *sb, /* in UTF-8 */
while (p > sb->buf && ch - p < 10 && *p != '\033')
p--;
if (*p != '\033' ||
- ch + 1 - p != display_mode_esc_sequence_len(p))
+ ch + 1 - p != display_mode_esc_sequence_len(p,
+ ch + 1 - p))
break;
/*
* got a good ansi sequence, put it back to
diff --git a/utf8.c b/utf8.c
index 96460cc414..544c5329f3 100644
--- a/utf8.c
+++ b/utf8.c
@@ -16,16 +16,23 @@ struct interval {
ucs_char_t last;
};
-size_t display_mode_esc_sequence_len(const char *s)
+/*
+ * Return the length of the ANSI display mode escape sequence (e.g.,
+ * "\033[1;31m") that appears at the beginning of the "len" bytes
+ * of string "s", or 0 if the bytes do not begin with a complete
+ * such sequence. Never looks at s[len] or beyond.
+ */
+size_t display_mode_esc_sequence_len(const char *s, size_t len)
{
- const char *p = s;
- if (*p++ != '\033')
+ const char *p = s, *end = s + len;
+
+ if (end <= p || *p++ != '\033')
return 0;
- if (*p++ != '[')
+ if (end <= p || *p++ != '[')
return 0;
- while (isdigit(*p) || *p == ';')
+ while (p < end && (isdigit(*p) || *p == ';'))
p++;
- if (*p++ != 'm')
+ if (end <= p || *p++ != 'm')
return 0;
return p - s;
}
@@ -218,7 +225,8 @@ int utf8_strnwidth(const char *string, size_t len, int skip_ansi)
size_t skip;
while (skip_ansi &&
- (skip = display_mode_esc_sequence_len(string)) != 0)
+ (skip = display_mode_esc_sequence_len(string,
+ orig + len - string)) != 0)
string += skip;
glyph_width = utf8_width(&string, NULL);
@@ -279,6 +287,7 @@ void strbuf_add_wrapped_text(struct strbuf *buf,
{
int indent, w, assume_utf8 = 1;
const char *bol, *space, *start = text;
+ const char *end = text + strlen(text);
size_t orig_len = buf->len;
if (width <= 0) {
@@ -299,7 +308,7 @@ void strbuf_add_wrapped_text(struct strbuf *buf,
char c;
size_t skip;
- while ((skip = display_mode_esc_sequence_len(text)))
+ while ((skip = display_mode_esc_sequence_len(text, end - text)))
text += skip;
c = *text;
@@ -378,7 +387,7 @@ void strbuf_utf8_replace(struct strbuf *sb_src, int pos, int width,
int glyph_width;
size_t n;
- while ((n = display_mode_esc_sequence_len(src))) {
+ while ((n = display_mode_esc_sequence_len(src, end - src))) {
strbuf_add(&dst, src, n);
src += n;
}
diff --git a/utf8.h b/utf8.h
index cf8ecb0f21..fb2227a9e2 100644
--- a/utf8.h
+++ b/utf8.h
@@ -5,7 +5,7 @@ struct strbuf;
typedef unsigned int ucs_char_t; /* assuming 32bit int */
-size_t display_mode_esc_sequence_len(const char *s);
+size_t display_mode_esc_sequence_len(const char *s, size_t len);
int utf8_width(const char **start, size_t *remainder_p);
int utf8_strnwidth(const char *string, size_t len, int skip_ansi);
int utf8_strwidth(const char *string);
--
2.56.1-740-gf38658f3d0
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v2] utf8: add display_mode_esc_sequence_len_counted()
2026-11-17 16:50 [PATCH] utf8: make display_mode_esc_sequence_len() take a length Junio C Hamano
@ 2026-11-17 20:06 ` Junio C Hamano
2026-11-17 21:26 ` Dirk Gouders
2026-11-17 23:08 ` Andy Koppe
0 siblings, 2 replies; 11+ messages in thread
From: Junio C Hamano @ 2026-11-17 20:06 UTC (permalink / raw)
To: git; +Cc: Dirk Gouders
display_mode_esc_sequence_len() scans for an ANSI display mode
escape sequence (e.g. "ESC [ 1 ; 31 m") at the start of a string,
relying on NUL termination to stop. Some callers, however, work on
a counted buffer: utf8_strnwidth() takes <string, len> and must not
scan past it, while strbuf_utf8_replace() and
strbuf_add_wrapped_text() know (or can cheaply learn) the end.
Add display_mode_esc_sequence_len_counted() taking <ptr, len>,
which never reads at or beyond ptr[len], and make
display_mode_esc_sequence_len() a thin wrapper for NUL-terminated
strings.
Use the counted variant in utf8_strnwidth() for correctness, and in
strbuf_utf8_replace() and strbuf_add_wrapped_text() (computing the
end once upfront in the latter) to avoid measuring the remaining
text in each iteration.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
* As there are new patches that want to add new callers to this
function, let's keep the original with the same API while adding
a new helper function that allows us to limit the scan.
utf8.c | 35 ++++++++++++++++++++++++++---------
utf8.h | 1 +
2 files changed, 27 insertions(+), 9 deletions(-)
diff --git a/utf8.c b/utf8.c
index 96460cc414..d0546ab31f 100644
--- a/utf8.c
+++ b/utf8.c
@@ -16,20 +16,33 @@ struct interval {
ucs_char_t last;
};
-size_t display_mode_esc_sequence_len(const char *s)
+/*
+ * Return the length of the ANSI display mode escape sequence (e.g.,
+ * "\033[1;31m") that appears at the beginning of the "len" bytes
+ * of string "s", or 0 if the bytes do not begin with a complete
+ * such sequence. Never looks at s[len] or beyond.
+ */
+size_t display_mode_esc_sequence_len_counted(const char *s, size_t len)
{
- const char *p = s;
- if (*p++ != '\033')
+ const char *p = s, *end = s + len;
+
+ if (end <= p || *p++ != '\033')
return 0;
- if (*p++ != '[')
+ if (end <= p || *p++ != '[')
return 0;
- while (isdigit(*p) || *p == ';')
+ while (p < end && (isdigit(*p) || *p == ';'))
p++;
- if (*p++ != 'm')
+ if (end <= p || *p++ != 'm')
return 0;
return p - s;
}
+/* Same as above, but for a NUL-terminated string "s". */
+size_t display_mode_esc_sequence_len(const char *s)
+{
+ return display_mode_esc_sequence_len_counted(s, strlen(s));
+}
+
/* auxiliary function for binary search in interval table */
static int bisearch(ucs_char_t ucs, const struct interval *table, int max)
{
@@ -218,7 +231,8 @@ int utf8_strnwidth(const char *string, size_t len, int skip_ansi)
size_t skip;
while (skip_ansi &&
- (skip = display_mode_esc_sequence_len(string)) != 0)
+ (skip = display_mode_esc_sequence_len_counted(string,
+ orig + len - string)) != 0)
string += skip;
glyph_width = utf8_width(&string, NULL);
@@ -279,6 +293,7 @@ void strbuf_add_wrapped_text(struct strbuf *buf,
{
int indent, w, assume_utf8 = 1;
const char *bol, *space, *start = text;
+ const char *end = text + strlen(text);
size_t orig_len = buf->len;
if (width <= 0) {
@@ -299,7 +314,8 @@ void strbuf_add_wrapped_text(struct strbuf *buf,
char c;
size_t skip;
- while ((skip = display_mode_esc_sequence_len(text)))
+ while ((skip = display_mode_esc_sequence_len_counted(text,
+ end - text)))
text += skip;
c = *text;
@@ -378,7 +394,8 @@ void strbuf_utf8_replace(struct strbuf *sb_src, int pos, int width,
int glyph_width;
size_t n;
- while ((n = display_mode_esc_sequence_len(src))) {
+ while ((n = display_mode_esc_sequence_len_counted(src,
+ end - src))) {
strbuf_add(&dst, src, n);
src += n;
}
diff --git a/utf8.h b/utf8.h
index cf8ecb0f21..57ca2ab270 100644
--- a/utf8.h
+++ b/utf8.h
@@ -6,6 +6,7 @@ struct strbuf;
typedef unsigned int ucs_char_t; /* assuming 32bit int */
size_t display_mode_esc_sequence_len(const char *s);
+size_t display_mode_esc_sequence_len_counted(const char *s, size_t len);
int utf8_width(const char **start, size_t *remainder_p);
int utf8_strnwidth(const char *string, size_t len, int skip_ansi);
int utf8_strwidth(const char *string);
--
2.56.1-740-gf38658f3d0
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted()
2026-11-17 20:06 ` [PATCH v2] utf8: add display_mode_esc_sequence_len_counted() Junio C Hamano
@ 2026-11-17 21:26 ` Dirk Gouders
2026-11-17 21:43 ` Junio C Hamano
2026-11-17 23:08 ` Andy Koppe
1 sibling, 1 reply; 11+ messages in thread
From: Dirk Gouders @ 2026-11-17 21:26 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
On Tue, Nov 17 2026, Junio C Hamano wrote:
> display_mode_esc_sequence_len() scans for an ANSI display mode
> escape sequence (e.g. "ESC [ 1 ; 31 m") at the start of a string,
> relying on NUL termination to stop. Some callers, however, work on
> a counted buffer: utf8_strnwidth() takes <string, len> and must not
> scan past it, while strbuf_utf8_replace() and
> strbuf_add_wrapped_text() know (or can cheaply learn) the end.
>
> Add display_mode_esc_sequence_len_counted() taking <ptr, len>,
> which never reads at or beyond ptr[len], and make
> display_mode_esc_sequence_len() a thin wrapper for NUL-terminated
> strings.
>
> Use the counted variant in utf8_strnwidth() for correctness, and in
> strbuf_utf8_replace() and strbuf_add_wrapped_text() (computing the
> end once upfront in the latter) to avoid measuring the remaining
> text in each iteration.
>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
> * As there are new patches that want to add new callers to this
> function, let's keep the original with the same API while adding
> a new helper function that allows us to limit the scan.
>
> utf8.c | 35 ++++++++++++++++++++++++++---------
> utf8.h | 1 +
> 2 files changed, 27 insertions(+), 9 deletions(-)
>
> diff --git a/utf8.c b/utf8.c
> index 96460cc414..d0546ab31f 100644
> --- a/utf8.c
> +++ b/utf8.c
> @@ -16,20 +16,33 @@ struct interval {
> ucs_char_t last;
> };
>
> -size_t display_mode_esc_sequence_len(const char *s)
> +/*
> + * Return the length of the ANSI display mode escape sequence (e.g.,
> + * "\033[1;31m") that appears at the beginning of the "len" bytes
> + * of string "s", or 0 if the bytes do not begin with a complete
> + * such sequence. Never looks at s[len] or beyond.
> + */
> +size_t display_mode_esc_sequence_len_counted(const char *s, size_t len)
> {
> - const char *p = s;
> - if (*p++ != '\033')
> + const char *p = s, *end = s + len;
You have better knowledge of the users of this function: would an early
return perhaps handle a significant amount of cases?
/* The shortest SGR sequence is "\033[m". */
if (len < 3)
return 0;
And, why not add some tests (the untested function I showed had
an off-by-one error):
diff --git a/Makefile b/Makefile
index 4c84eea6e2..3c8dc9ca40 100644
--- a/Makefile
+++ b/Makefile
@@ -1564,6 +1564,7 @@ CLAR_TEST_SUITES += u-strvec
CLAR_TEST_SUITES += u-trailer
CLAR_TEST_SUITES += u-urlmatch-normalization
CLAR_TEST_SUITES += u-utf8-width
+CLAR_TEST_SUITES += u-utf8-ansi
CLAR_TEST_PROG = $(UNIT_TEST_BIN)/unit-tests$(X)
CLAR_TEST_OBJS = $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(CLAR_TEST_SUITES))
CLAR_TEST_OBJS += $(UNIT_TEST_DIR)/clar/clar.o
diff --git a/t/meson.build b/t/meson.build
index 1cf1d95f70..e224df58c8 100644
--- a/t/meson.build
+++ b/t/meson.build
@@ -27,6 +27,7 @@ clar_test_suites = [
'unit-tests/u-trailer.c',
'unit-tests/u-urlmatch-normalization.c',
'unit-tests/u-utf8-width.c',
+ 'unit-tests/u-utf8-ansi.c',
]
clar_sources = [
diff --git a/t/unit-tests/u-utf8-ansi.c b/t/unit-tests/u-utf8-ansi.c
new file mode 100644
index 0000000000..c19d8bce78
--- /dev/null
+++ b/t/unit-tests/u-utf8-ansi.c
@@ -0,0 +1,14 @@
+#include "unit-test.h"
+#include "utf8.h"
+#include "strbuf.h"
+
+/*
+ * Test len limited functions for ANSI SGR lengths.
+ */
+void test_utf8_ansi__sgr_len(void)
+{
+ cl_assert_equal_i(0, display_mode_esc_sequence_len_counted("\033[m", 2));
+ cl_assert_equal_i(3, display_mode_esc_sequence_len_counted("\033[m", 3));
+ cl_assert_equal_i(5, display_mode_esc_sequence_len_counted("\033[31m", 5));
+ cl_assert_equal_i(7, display_mode_esc_sequence_len_counted("\033[0;31m", 7));
+}
Regards,
Dirk
> + if (end <= p || *p++ != '\033')
> return 0;
> - if (*p++ != '[')
> + if (end <= p || *p++ != '[')
> return 0;
> - while (isdigit(*p) || *p == ';')
> + while (p < end && (isdigit(*p) || *p == ';'))
> p++;
> - if (*p++ != 'm')
> + if (end <= p || *p++ != 'm')
> return 0;
> return p - s;
> }
>
> +/* Same as above, but for a NUL-terminated string "s". */
> +size_t display_mode_esc_sequence_len(const char *s)
> +{
> + return display_mode_esc_sequence_len_counted(s, strlen(s));
> +}
> +
> /* auxiliary function for binary search in interval table */
> static int bisearch(ucs_char_t ucs, const struct interval *table, int max)
> {
> @@ -218,7 +231,8 @@ int utf8_strnwidth(const char *string, size_t len, int skip_ansi)
> size_t skip;
>
> while (skip_ansi &&
> - (skip = display_mode_esc_sequence_len(string)) != 0)
> + (skip = display_mode_esc_sequence_len_counted(string,
> + orig + len - string)) != 0)
> string += skip;
>
> glyph_width = utf8_width(&string, NULL);
> @@ -279,6 +293,7 @@ void strbuf_add_wrapped_text(struct strbuf *buf,
> {
> int indent, w, assume_utf8 = 1;
> const char *bol, *space, *start = text;
> + const char *end = text + strlen(text);
> size_t orig_len = buf->len;
>
> if (width <= 0) {
> @@ -299,7 +314,8 @@ void strbuf_add_wrapped_text(struct strbuf *buf,
> char c;
> size_t skip;
>
> - while ((skip = display_mode_esc_sequence_len(text)))
> + while ((skip = display_mode_esc_sequence_len_counted(text,
> + end - text)))
> text += skip;
>
> c = *text;
> @@ -378,7 +394,8 @@ void strbuf_utf8_replace(struct strbuf *sb_src, int pos, int width,
> int glyph_width;
> size_t n;
>
> - while ((n = display_mode_esc_sequence_len(src))) {
> + while ((n = display_mode_esc_sequence_len_counted(src,
> + end - src))) {
> strbuf_add(&dst, src, n);
> src += n;
> }
> diff --git a/utf8.h b/utf8.h
> index cf8ecb0f21..57ca2ab270 100644
> --- a/utf8.h
> +++ b/utf8.h
> @@ -6,6 +6,7 @@ struct strbuf;
> typedef unsigned int ucs_char_t; /* assuming 32bit int */
>
> size_t display_mode_esc_sequence_len(const char *s);
> +size_t display_mode_esc_sequence_len_counted(const char *s, size_t len);
> int utf8_width(const char **start, size_t *remainder_p);
> int utf8_strnwidth(const char *string, size_t len, int skip_ansi);
> int utf8_strwidth(const char *string);
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted()
2026-11-17 21:26 ` Dirk Gouders
@ 2026-11-17 21:43 ` Junio C Hamano
2026-11-17 22:03 ` Dirk Gouders
0 siblings, 1 reply; 11+ messages in thread
From: Junio C Hamano @ 2026-11-17 21:43 UTC (permalink / raw)
To: Dirk Gouders; +Cc: git
Dirk Gouders <dirk@gouders.net> writes:
> You have better knowledge of the users of this function: would an early
> return perhaps handle a significant amount of cases?
>
> /* The shortest SGR sequence is "\033[m". */
> if (len < 3)
> return 0;
Do you have any benchmarks? I don't.
I find it of dubious performance merit to run this comparison for a
long string as it is eaten one UTF-8 characters at a time by the
caller, and when the length is actually shorter than 3, then
runnning the remainder of the function for a few steps should not be
10x more expensive than the above comparison, no?
> And, why not add some tests (the untested function I showed had
> an off-by-one error):
Sounds good. Should I just steal what you sent out?
> diff --git a/Makefile b/Makefile
> index 4c84eea6e2..3c8dc9ca40 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1564,6 +1564,7 @@ CLAR_TEST_SUITES += u-strvec
> CLAR_TEST_SUITES += u-trailer
> CLAR_TEST_SUITES += u-urlmatch-normalization
> CLAR_TEST_SUITES += u-utf8-width
> +CLAR_TEST_SUITES += u-utf8-ansi
> CLAR_TEST_PROG = $(UNIT_TEST_BIN)/unit-tests$(X)
> CLAR_TEST_OBJS = $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(CLAR_TEST_SUITES))
> CLAR_TEST_OBJS += $(UNIT_TEST_DIR)/clar/clar.o
> diff --git a/t/meson.build b/t/meson.build
> index 1cf1d95f70..e224df58c8 100644
> --- a/t/meson.build
> +++ b/t/meson.build
> @@ -27,6 +27,7 @@ clar_test_suites = [
> 'unit-tests/u-trailer.c',
> 'unit-tests/u-urlmatch-normalization.c',
> 'unit-tests/u-utf8-width.c',
> + 'unit-tests/u-utf8-ansi.c',
> ]
>
> clar_sources = [
> diff --git a/t/unit-tests/u-utf8-ansi.c b/t/unit-tests/u-utf8-ansi.c
> new file mode 100644
> index 0000000000..c19d8bce78
> --- /dev/null
> +++ b/t/unit-tests/u-utf8-ansi.c
> @@ -0,0 +1,14 @@
> +#include "unit-test.h"
> +#include "utf8.h"
> +#include "strbuf.h"
> +
> +/*
> + * Test len limited functions for ANSI SGR lengths.
> + */
> +void test_utf8_ansi__sgr_len(void)
> +{
> + cl_assert_equal_i(0, display_mode_esc_sequence_len_counted("\033[m", 2));
> + cl_assert_equal_i(3, display_mode_esc_sequence_len_counted("\033[m", 3));
> + cl_assert_equal_i(5, display_mode_esc_sequence_len_counted("\033[31m", 5));
> + cl_assert_equal_i(7, display_mode_esc_sequence_len_counted("\033[0;31m", 7));
> +}
Thanks.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted()
2026-11-17 21:43 ` Junio C Hamano
@ 2026-11-17 22:03 ` Dirk Gouders
2026-11-18 1:33 ` Junio C Hamano
0 siblings, 1 reply; 11+ messages in thread
From: Dirk Gouders @ 2026-11-17 22:03 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
On Tue, Nov 17 2026, Junio C Hamano wrote:
> Dirk Gouders <dirk@gouders.net> writes:
>
>> You have better knowledge of the users of this function: would an early
>> return perhaps handle a significant amount of cases?
>>
>> /* The shortest SGR sequence is "\033[m". */
>> if (len < 3)
>> return 0;
>
> Do you have any benchmarks? I don't.
>
> I find it of dubious performance merit to run this comparison for a
> long string as it is eaten one UTF-8 characters at a time by the
> caller, and when the length is actually shorter than 3, then
> runnning the remainder of the function for a few steps should not be
> 10x more expensive than the above comparison, no?
Meanwhile, I thought: even if the test would be a good idea, the
"\033"-test probably is the one that handles most cases.
Maybe I will look closer, but not now. Correctness is more important
and I need to work on v2.
>> And, why not add some tests (the untested function I showed had
>> an off-by-one error):
>
> Sounds good. Should I just steal what you sent out?
That wouldn't be stealing, of course you can use it.
Regards,
Dirk
>> diff --git a/Makefile b/Makefile
>> index 4c84eea6e2..3c8dc9ca40 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -1564,6 +1564,7 @@ CLAR_TEST_SUITES += u-strvec
>> CLAR_TEST_SUITES += u-trailer
>> CLAR_TEST_SUITES += u-urlmatch-normalization
>> CLAR_TEST_SUITES += u-utf8-width
>> +CLAR_TEST_SUITES += u-utf8-ansi
>> CLAR_TEST_PROG = $(UNIT_TEST_BIN)/unit-tests$(X)
>> CLAR_TEST_OBJS = $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(CLAR_TEST_SUITES))
>> CLAR_TEST_OBJS += $(UNIT_TEST_DIR)/clar/clar.o
>> diff --git a/t/meson.build b/t/meson.build
>> index 1cf1d95f70..e224df58c8 100644
>> --- a/t/meson.build
>> +++ b/t/meson.build
>> @@ -27,6 +27,7 @@ clar_test_suites = [
>> 'unit-tests/u-trailer.c',
>> 'unit-tests/u-urlmatch-normalization.c',
>> 'unit-tests/u-utf8-width.c',
>> + 'unit-tests/u-utf8-ansi.c',
>> ]
>>
>> clar_sources = [
>> diff --git a/t/unit-tests/u-utf8-ansi.c b/t/unit-tests/u-utf8-ansi.c
>> new file mode 100644
>> index 0000000000..c19d8bce78
>> --- /dev/null
>> +++ b/t/unit-tests/u-utf8-ansi.c
>> @@ -0,0 +1,14 @@
>> +#include "unit-test.h"
>> +#include "utf8.h"
>> +#include "strbuf.h"
>> +
>> +/*
>> + * Test len limited functions for ANSI SGR lengths.
>> + */
>> +void test_utf8_ansi__sgr_len(void)
>> +{
>> + cl_assert_equal_i(0, display_mode_esc_sequence_len_counted("\033[m", 2));
>> + cl_assert_equal_i(3, display_mode_esc_sequence_len_counted("\033[m", 3));
>> + cl_assert_equal_i(5, display_mode_esc_sequence_len_counted("\033[31m", 5));
>> + cl_assert_equal_i(7, display_mode_esc_sequence_len_counted("\033[0;31m", 7));
>> +}
>
> Thanks.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted()
2026-11-17 20:06 ` [PATCH v2] utf8: add display_mode_esc_sequence_len_counted() Junio C Hamano
2026-11-17 21:26 ` Dirk Gouders
@ 2026-11-17 23:08 ` Andy Koppe
2026-11-17 23:25 ` Junio C Hamano
1 sibling, 1 reply; 11+ messages in thread
From: Andy Koppe @ 2026-11-17 23:08 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Dirk Gouders
Junio C Hamano wrote:
> - while (isdigit(*p) || *p == ';')
> + while (p < end && (isdigit(*p) || *p == ';'))
While you're there, could you add colons (':') to that?
ECMA-48 §5.4.2 defines them as sub-separators within the
semicolon-separated parameters of control sequences. Colons are
referred to as "3/10" there, aka ASCII 0x3A:
https://ecma-international.org/wp-content/uploads/ECMA-48_5th_edition_june_1991.pdf#page=26
Maybe more practically relevant, xterm and its many
more-or-less-compatible brethren accept colons in SGR sequences:
"xterm allows either colons (standard) or semicolons (legacy) to
separate the subparameters (but after the first colon, colons must be
used)"
https://invisible-island.net/xterm/ctlseqs/ctlseqs.html
Andy
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted()
2026-11-17 23:08 ` Andy Koppe
@ 2026-11-17 23:25 ` Junio C Hamano
2026-11-18 6:59 ` Andy Koppe
0 siblings, 1 reply; 11+ messages in thread
From: Junio C Hamano @ 2026-11-17 23:25 UTC (permalink / raw)
To: Andy Koppe; +Cc: git, Dirk Gouders
Andy Koppe <andy.koppe@gmail.com> writes:
> Junio C Hamano wrote:
>
>> - while (isdigit(*p) || *p == ';')
>> + while (p < end && (isdigit(*p) || *p == ';'))
>
> While you're there, could you add colons (':') to that?
Do we generate such a sequence that uses subseparators?
IIRC, this code is not counting the display width of end-user
supplied string. Rather, it gets string we paint in a buffer and
then counts how much display column it occupies.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted()
2026-11-17 22:03 ` Dirk Gouders
@ 2026-11-18 1:33 ` Junio C Hamano
2026-11-18 7:37 ` Dirk Gouders
0 siblings, 1 reply; 11+ messages in thread
From: Junio C Hamano @ 2026-11-18 1:33 UTC (permalink / raw)
To: Dirk Gouders; +Cc: git
Dirk Gouders <dirk@gouders.net> writes:
> Meanwhile, I thought: even if the test would be a good idea, the
> "\033"-test probably is the one that handles most cases.
>
> Maybe I will look closer, but not now. Correctness is more important
> and I need to work on v2.
It would be nice if you can take this patch over and make it your
[PATCH v2 1/2].
Thanks.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted()
2026-11-17 23:25 ` Junio C Hamano
@ 2026-11-18 6:59 ` Andy Koppe
2026-11-18 7:43 ` Dirk Gouders
0 siblings, 1 reply; 11+ messages in thread
From: Andy Koppe @ 2026-11-18 6:59 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Dirk Gouders
Junio C Hamano wrote:
>
> Andy Koppe <andy.koppe@gmail.com> writes:
>
> > Junio C Hamano wrote:
> >
> >> - while (isdigit(*p) || *p == ';')
> >> + while (p < end && (isdigit(*p) || *p == ';'))
> >
> > While you're there, could you add colons (':') to that?
>
> Do we generate such a sequence that uses subseparators?
>
> IIRC, this code is not counting the display width of end-user
> supplied string. Rather, it gets string we paint in a buffer and
> then counts how much display column it occupies.
You're right that Git itself doesn't generate them (although future
enhancements might change that).
However, display_mode_esc_sequence_len() already operates on expanded
custom formats and message text, e.g. in rewrap_message_tail() via
strbuf_wrap(). And on stdin in git-column. They could all contain SGR
sequences with colons, whose width currently isn't ignored as
intended. (And my proposed make_line_colors_self_contained() wouldn't
reset or replay them.)
The check could be changed to `*p >= '0' && *p<=';'` to cheaply include colons.
Andy
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted()
2026-11-18 1:33 ` Junio C Hamano
@ 2026-11-18 7:37 ` Dirk Gouders
0 siblings, 0 replies; 11+ messages in thread
From: Dirk Gouders @ 2026-11-18 7:37 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
On Tue, Nov 17 2026, Junio C Hamano wrote:
> Dirk Gouders <dirk@gouders.net> writes:
>
>> Meanwhile, I thought: even if the test would be a good idea, the
>> "\033"-test probably is the one that handles most cases.
>>
>> Maybe I will look closer, but not now. Correctness is more important
>> and I need to work on v2.
>
> It would be nice if you can take this patch over and make it your
> [PATCH v2 1/2].
OK, I will do so, thank you.
Regards,
Dirk
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted()
2026-11-18 6:59 ` Andy Koppe
@ 2026-11-18 7:43 ` Dirk Gouders
0 siblings, 0 replies; 11+ messages in thread
From: Dirk Gouders @ 2026-11-18 7:43 UTC (permalink / raw)
To: Andy Koppe; +Cc: Junio C Hamano, git
On Wed, Nov 18 2026, Andy Koppe wrote:
> Junio C Hamano wrote:
>>
>> Andy Koppe <andy.koppe@gmail.com> writes:
>>
>> > Junio C Hamano wrote:
>> >
>> >> - while (isdigit(*p) || *p == ';')
>> >> + while (p < end && (isdigit(*p) || *p == ';'))
>> >
>> > While you're there, could you add colons (':') to that?
>>
>> Do we generate such a sequence that uses subseparators?
>>
>> IIRC, this code is not counting the display width of end-user
>> supplied string. Rather, it gets string we paint in a buffer and
>> then counts how much display column it occupies.
>
> You're right that Git itself doesn't generate them (although future
> enhancements might change that).
>
> However, display_mode_esc_sequence_len() already operates on expanded
> custom formats and message text, e.g. in rewrap_message_tail() via
> strbuf_wrap(). And on stdin in git-column. They could all contain SGR
> sequences with colons, whose width currently isn't ignored as
> intended. (And my proposed make_line_colors_self_contained() wouldn't
> reset or replay them.)
>
> The check could be changed to `*p >= '0' && *p<=';'` to cheaply include colons.
I would suggest to not mix these changes simply for bisection purposes.
And as far as I know, we would need an FSM to identify valid sequences
if we are going to allow colons.
Regards,
Dirk
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2026-11-18 7:43 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-17 16:50 [PATCH] utf8: make display_mode_esc_sequence_len() take a length Junio C Hamano
2026-11-17 20:06 ` [PATCH v2] utf8: add display_mode_esc_sequence_len_counted() Junio C Hamano
2026-11-17 21:26 ` Dirk Gouders
2026-11-17 21:43 ` Junio C Hamano
2026-11-17 22:03 ` Dirk Gouders
2026-11-18 1:33 ` Junio C Hamano
2026-11-18 7:37 ` Dirk Gouders
2026-11-17 23:08 ` Andy Koppe
2026-11-17 23:25 ` Junio C Hamano
2026-11-18 6:59 ` Andy Koppe
2026-11-18 7:43 ` Dirk Gouders
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox