* [PATCH] utf8: make display_mode_esc_sequence_len() take a length @ 2026-11-17 16:50 Junio C Hamano 2026-11-17 20:06 ` [PATCH v2] utf8: add display_mode_esc_sequence_len_counted() Junio C Hamano 0 siblings, 1 reply; 11+ messages in thread From: Junio C Hamano @ 2026-11-17 16:50 UTC (permalink / raw) To: git; +Cc: Dirk Gouders display_mode_esc_sequence_len() scans for an ANSI display mode escape sequence (e.g. "ESC [ 1 ; 31 m"), relying on NUL termination to stop. Some callers, however, use a counted <ptr, len> buffer or need to stop at a specific position. Teach it to take a buffer length and never read at or beyond ptr[len], returning the length of a complete sequence at the start or 0 otherwise. Update the callers to pass the remaining byte count. For strbuf_add_wrapped_text(), which takes a NUL-terminated string, compute the end once upfront instead of calling strlen() each time. Signed-off-by: Junio C Hamano <gitster@pobox.com> --- * Copy of https://lore.kernel.org/git/xmqqse0zfn2n.fsf@gitster.g/ with a proposed commit log message. pretty.c | 3 ++- utf8.c | 27 ++++++++++++++++++--------- utf8.h | 2 +- 3 files changed, 21 insertions(+), 11 deletions(-) diff --git a/pretty.c b/pretty.c index 0bc8ad8a9a..a20fd61909 100644 --- a/pretty.c +++ b/pretty.c @@ -1834,7 +1834,8 @@ static size_t format_and_pad_commit(struct strbuf *sb, /* in UTF-8 */ while (p > sb->buf && ch - p < 10 && *p != '\033') p--; if (*p != '\033' || - ch + 1 - p != display_mode_esc_sequence_len(p)) + ch + 1 - p != display_mode_esc_sequence_len(p, + ch + 1 - p)) break; /* * got a good ansi sequence, put it back to diff --git a/utf8.c b/utf8.c index 96460cc414..544c5329f3 100644 --- a/utf8.c +++ b/utf8.c @@ -16,16 +16,23 @@ struct interval { ucs_char_t last; }; -size_t display_mode_esc_sequence_len(const char *s) +/* + * Return the length of the ANSI display mode escape sequence (e.g., + * "\033[1;31m") that appears at the beginning of the "len" bytes + * of string "s", or 0 if the bytes do not begin with a complete + * such sequence. Never looks at s[len] or beyond. + */ +size_t display_mode_esc_sequence_len(const char *s, size_t len) { - const char *p = s; - if (*p++ != '\033') + const char *p = s, *end = s + len; + + if (end <= p || *p++ != '\033') return 0; - if (*p++ != '[') + if (end <= p || *p++ != '[') return 0; - while (isdigit(*p) || *p == ';') + while (p < end && (isdigit(*p) || *p == ';')) p++; - if (*p++ != 'm') + if (end <= p || *p++ != 'm') return 0; return p - s; } @@ -218,7 +225,8 @@ int utf8_strnwidth(const char *string, size_t len, int skip_ansi) size_t skip; while (skip_ansi && - (skip = display_mode_esc_sequence_len(string)) != 0) + (skip = display_mode_esc_sequence_len(string, + orig + len - string)) != 0) string += skip; glyph_width = utf8_width(&string, NULL); @@ -279,6 +287,7 @@ void strbuf_add_wrapped_text(struct strbuf *buf, { int indent, w, assume_utf8 = 1; const char *bol, *space, *start = text; + const char *end = text + strlen(text); size_t orig_len = buf->len; if (width <= 0) { @@ -299,7 +308,7 @@ void strbuf_add_wrapped_text(struct strbuf *buf, char c; size_t skip; - while ((skip = display_mode_esc_sequence_len(text))) + while ((skip = display_mode_esc_sequence_len(text, end - text))) text += skip; c = *text; @@ -378,7 +387,7 @@ void strbuf_utf8_replace(struct strbuf *sb_src, int pos, int width, int glyph_width; size_t n; - while ((n = display_mode_esc_sequence_len(src))) { + while ((n = display_mode_esc_sequence_len(src, end - src))) { strbuf_add(&dst, src, n); src += n; } diff --git a/utf8.h b/utf8.h index cf8ecb0f21..fb2227a9e2 100644 --- a/utf8.h +++ b/utf8.h @@ -5,7 +5,7 @@ struct strbuf; typedef unsigned int ucs_char_t; /* assuming 32bit int */ -size_t display_mode_esc_sequence_len(const char *s); +size_t display_mode_esc_sequence_len(const char *s, size_t len); int utf8_width(const char **start, size_t *remainder_p); int utf8_strnwidth(const char *string, size_t len, int skip_ansi); int utf8_strwidth(const char *string); -- 2.56.1-740-gf38658f3d0 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v2] utf8: add display_mode_esc_sequence_len_counted() 2026-11-17 16:50 [PATCH] utf8: make display_mode_esc_sequence_len() take a length Junio C Hamano @ 2026-11-17 20:06 ` Junio C Hamano 2026-11-17 21:26 ` Dirk Gouders 2026-11-17 23:08 ` Andy Koppe 0 siblings, 2 replies; 11+ messages in thread From: Junio C Hamano @ 2026-11-17 20:06 UTC (permalink / raw) To: git; +Cc: Dirk Gouders display_mode_esc_sequence_len() scans for an ANSI display mode escape sequence (e.g. "ESC [ 1 ; 31 m") at the start of a string, relying on NUL termination to stop. Some callers, however, work on a counted buffer: utf8_strnwidth() takes <string, len> and must not scan past it, while strbuf_utf8_replace() and strbuf_add_wrapped_text() know (or can cheaply learn) the end. Add display_mode_esc_sequence_len_counted() taking <ptr, len>, which never reads at or beyond ptr[len], and make display_mode_esc_sequence_len() a thin wrapper for NUL-terminated strings. Use the counted variant in utf8_strnwidth() for correctness, and in strbuf_utf8_replace() and strbuf_add_wrapped_text() (computing the end once upfront in the latter) to avoid measuring the remaining text in each iteration. Signed-off-by: Junio C Hamano <gitster@pobox.com> --- * As there are new patches that want to add new callers to this function, let's keep the original with the same API while adding a new helper function that allows us to limit the scan. utf8.c | 35 ++++++++++++++++++++++++++--------- utf8.h | 1 + 2 files changed, 27 insertions(+), 9 deletions(-) diff --git a/utf8.c b/utf8.c index 96460cc414..d0546ab31f 100644 --- a/utf8.c +++ b/utf8.c @@ -16,20 +16,33 @@ struct interval { ucs_char_t last; }; -size_t display_mode_esc_sequence_len(const char *s) +/* + * Return the length of the ANSI display mode escape sequence (e.g., + * "\033[1;31m") that appears at the beginning of the "len" bytes + * of string "s", or 0 if the bytes do not begin with a complete + * such sequence. Never looks at s[len] or beyond. + */ +size_t display_mode_esc_sequence_len_counted(const char *s, size_t len) { - const char *p = s; - if (*p++ != '\033') + const char *p = s, *end = s + len; + + if (end <= p || *p++ != '\033') return 0; - if (*p++ != '[') + if (end <= p || *p++ != '[') return 0; - while (isdigit(*p) || *p == ';') + while (p < end && (isdigit(*p) || *p == ';')) p++; - if (*p++ != 'm') + if (end <= p || *p++ != 'm') return 0; return p - s; } +/* Same as above, but for a NUL-terminated string "s". */ +size_t display_mode_esc_sequence_len(const char *s) +{ + return display_mode_esc_sequence_len_counted(s, strlen(s)); +} + /* auxiliary function for binary search in interval table */ static int bisearch(ucs_char_t ucs, const struct interval *table, int max) { @@ -218,7 +231,8 @@ int utf8_strnwidth(const char *string, size_t len, int skip_ansi) size_t skip; while (skip_ansi && - (skip = display_mode_esc_sequence_len(string)) != 0) + (skip = display_mode_esc_sequence_len_counted(string, + orig + len - string)) != 0) string += skip; glyph_width = utf8_width(&string, NULL); @@ -279,6 +293,7 @@ void strbuf_add_wrapped_text(struct strbuf *buf, { int indent, w, assume_utf8 = 1; const char *bol, *space, *start = text; + const char *end = text + strlen(text); size_t orig_len = buf->len; if (width <= 0) { @@ -299,7 +314,8 @@ void strbuf_add_wrapped_text(struct strbuf *buf, char c; size_t skip; - while ((skip = display_mode_esc_sequence_len(text))) + while ((skip = display_mode_esc_sequence_len_counted(text, + end - text))) text += skip; c = *text; @@ -378,7 +394,8 @@ void strbuf_utf8_replace(struct strbuf *sb_src, int pos, int width, int glyph_width; size_t n; - while ((n = display_mode_esc_sequence_len(src))) { + while ((n = display_mode_esc_sequence_len_counted(src, + end - src))) { strbuf_add(&dst, src, n); src += n; } diff --git a/utf8.h b/utf8.h index cf8ecb0f21..57ca2ab270 100644 --- a/utf8.h +++ b/utf8.h @@ -6,6 +6,7 @@ struct strbuf; typedef unsigned int ucs_char_t; /* assuming 32bit int */ size_t display_mode_esc_sequence_len(const char *s); +size_t display_mode_esc_sequence_len_counted(const char *s, size_t len); int utf8_width(const char **start, size_t *remainder_p); int utf8_strnwidth(const char *string, size_t len, int skip_ansi); int utf8_strwidth(const char *string); -- 2.56.1-740-gf38658f3d0 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted() 2026-11-17 20:06 ` [PATCH v2] utf8: add display_mode_esc_sequence_len_counted() Junio C Hamano @ 2026-11-17 21:26 ` Dirk Gouders 2026-11-17 21:43 ` Junio C Hamano 2026-11-17 23:08 ` Andy Koppe 1 sibling, 1 reply; 11+ messages in thread From: Dirk Gouders @ 2026-11-17 21:26 UTC (permalink / raw) To: Junio C Hamano; +Cc: git On Tue, Nov 17 2026, Junio C Hamano wrote: > display_mode_esc_sequence_len() scans for an ANSI display mode > escape sequence (e.g. "ESC [ 1 ; 31 m") at the start of a string, > relying on NUL termination to stop. Some callers, however, work on > a counted buffer: utf8_strnwidth() takes <string, len> and must not > scan past it, while strbuf_utf8_replace() and > strbuf_add_wrapped_text() know (or can cheaply learn) the end. > > Add display_mode_esc_sequence_len_counted() taking <ptr, len>, > which never reads at or beyond ptr[len], and make > display_mode_esc_sequence_len() a thin wrapper for NUL-terminated > strings. > > Use the counted variant in utf8_strnwidth() for correctness, and in > strbuf_utf8_replace() and strbuf_add_wrapped_text() (computing the > end once upfront in the latter) to avoid measuring the remaining > text in each iteration. > > Signed-off-by: Junio C Hamano <gitster@pobox.com> > --- > * As there are new patches that want to add new callers to this > function, let's keep the original with the same API while adding > a new helper function that allows us to limit the scan. > > utf8.c | 35 ++++++++++++++++++++++++++--------- > utf8.h | 1 + > 2 files changed, 27 insertions(+), 9 deletions(-) > > diff --git a/utf8.c b/utf8.c > index 96460cc414..d0546ab31f 100644 > --- a/utf8.c > +++ b/utf8.c > @@ -16,20 +16,33 @@ struct interval { > ucs_char_t last; > }; > > -size_t display_mode_esc_sequence_len(const char *s) > +/* > + * Return the length of the ANSI display mode escape sequence (e.g., > + * "\033[1;31m") that appears at the beginning of the "len" bytes > + * of string "s", or 0 if the bytes do not begin with a complete > + * such sequence. Never looks at s[len] or beyond. > + */ > +size_t display_mode_esc_sequence_len_counted(const char *s, size_t len) > { > - const char *p = s; > - if (*p++ != '\033') > + const char *p = s, *end = s + len; You have better knowledge of the users of this function: would an early return perhaps handle a significant amount of cases? /* The shortest SGR sequence is "\033[m". */ if (len < 3) return 0; And, why not add some tests (the untested function I showed had an off-by-one error): diff --git a/Makefile b/Makefile index 4c84eea6e2..3c8dc9ca40 100644 --- a/Makefile +++ b/Makefile @@ -1564,6 +1564,7 @@ CLAR_TEST_SUITES += u-strvec CLAR_TEST_SUITES += u-trailer CLAR_TEST_SUITES += u-urlmatch-normalization CLAR_TEST_SUITES += u-utf8-width +CLAR_TEST_SUITES += u-utf8-ansi CLAR_TEST_PROG = $(UNIT_TEST_BIN)/unit-tests$(X) CLAR_TEST_OBJS = $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(CLAR_TEST_SUITES)) CLAR_TEST_OBJS += $(UNIT_TEST_DIR)/clar/clar.o diff --git a/t/meson.build b/t/meson.build index 1cf1d95f70..e224df58c8 100644 --- a/t/meson.build +++ b/t/meson.build @@ -27,6 +27,7 @@ clar_test_suites = [ 'unit-tests/u-trailer.c', 'unit-tests/u-urlmatch-normalization.c', 'unit-tests/u-utf8-width.c', + 'unit-tests/u-utf8-ansi.c', ] clar_sources = [ diff --git a/t/unit-tests/u-utf8-ansi.c b/t/unit-tests/u-utf8-ansi.c new file mode 100644 index 0000000000..c19d8bce78 --- /dev/null +++ b/t/unit-tests/u-utf8-ansi.c @@ -0,0 +1,14 @@ +#include "unit-test.h" +#include "utf8.h" +#include "strbuf.h" + +/* + * Test len limited functions for ANSI SGR lengths. + */ +void test_utf8_ansi__sgr_len(void) +{ + cl_assert_equal_i(0, display_mode_esc_sequence_len_counted("\033[m", 2)); + cl_assert_equal_i(3, display_mode_esc_sequence_len_counted("\033[m", 3)); + cl_assert_equal_i(5, display_mode_esc_sequence_len_counted("\033[31m", 5)); + cl_assert_equal_i(7, display_mode_esc_sequence_len_counted("\033[0;31m", 7)); +} Regards, Dirk > + if (end <= p || *p++ != '\033') > return 0; > - if (*p++ != '[') > + if (end <= p || *p++ != '[') > return 0; > - while (isdigit(*p) || *p == ';') > + while (p < end && (isdigit(*p) || *p == ';')) > p++; > - if (*p++ != 'm') > + if (end <= p || *p++ != 'm') > return 0; > return p - s; > } > > +/* Same as above, but for a NUL-terminated string "s". */ > +size_t display_mode_esc_sequence_len(const char *s) > +{ > + return display_mode_esc_sequence_len_counted(s, strlen(s)); > +} > + > /* auxiliary function for binary search in interval table */ > static int bisearch(ucs_char_t ucs, const struct interval *table, int max) > { > @@ -218,7 +231,8 @@ int utf8_strnwidth(const char *string, size_t len, int skip_ansi) > size_t skip; > > while (skip_ansi && > - (skip = display_mode_esc_sequence_len(string)) != 0) > + (skip = display_mode_esc_sequence_len_counted(string, > + orig + len - string)) != 0) > string += skip; > > glyph_width = utf8_width(&string, NULL); > @@ -279,6 +293,7 @@ void strbuf_add_wrapped_text(struct strbuf *buf, > { > int indent, w, assume_utf8 = 1; > const char *bol, *space, *start = text; > + const char *end = text + strlen(text); > size_t orig_len = buf->len; > > if (width <= 0) { > @@ -299,7 +314,8 @@ void strbuf_add_wrapped_text(struct strbuf *buf, > char c; > size_t skip; > > - while ((skip = display_mode_esc_sequence_len(text))) > + while ((skip = display_mode_esc_sequence_len_counted(text, > + end - text))) > text += skip; > > c = *text; > @@ -378,7 +394,8 @@ void strbuf_utf8_replace(struct strbuf *sb_src, int pos, int width, > int glyph_width; > size_t n; > > - while ((n = display_mode_esc_sequence_len(src))) { > + while ((n = display_mode_esc_sequence_len_counted(src, > + end - src))) { > strbuf_add(&dst, src, n); > src += n; > } > diff --git a/utf8.h b/utf8.h > index cf8ecb0f21..57ca2ab270 100644 > --- a/utf8.h > +++ b/utf8.h > @@ -6,6 +6,7 @@ struct strbuf; > typedef unsigned int ucs_char_t; /* assuming 32bit int */ > > size_t display_mode_esc_sequence_len(const char *s); > +size_t display_mode_esc_sequence_len_counted(const char *s, size_t len); > int utf8_width(const char **start, size_t *remainder_p); > int utf8_strnwidth(const char *string, size_t len, int skip_ansi); > int utf8_strwidth(const char *string); ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted() 2026-11-17 21:26 ` Dirk Gouders @ 2026-11-17 21:43 ` Junio C Hamano 2026-11-17 22:03 ` Dirk Gouders 0 siblings, 1 reply; 11+ messages in thread From: Junio C Hamano @ 2026-11-17 21:43 UTC (permalink / raw) To: Dirk Gouders; +Cc: git Dirk Gouders <dirk@gouders.net> writes: > You have better knowledge of the users of this function: would an early > return perhaps handle a significant amount of cases? > > /* The shortest SGR sequence is "\033[m". */ > if (len < 3) > return 0; Do you have any benchmarks? I don't. I find it of dubious performance merit to run this comparison for a long string as it is eaten one UTF-8 characters at a time by the caller, and when the length is actually shorter than 3, then runnning the remainder of the function for a few steps should not be 10x more expensive than the above comparison, no? > And, why not add some tests (the untested function I showed had > an off-by-one error): Sounds good. Should I just steal what you sent out? > diff --git a/Makefile b/Makefile > index 4c84eea6e2..3c8dc9ca40 100644 > --- a/Makefile > +++ b/Makefile > @@ -1564,6 +1564,7 @@ CLAR_TEST_SUITES += u-strvec > CLAR_TEST_SUITES += u-trailer > CLAR_TEST_SUITES += u-urlmatch-normalization > CLAR_TEST_SUITES += u-utf8-width > +CLAR_TEST_SUITES += u-utf8-ansi > CLAR_TEST_PROG = $(UNIT_TEST_BIN)/unit-tests$(X) > CLAR_TEST_OBJS = $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(CLAR_TEST_SUITES)) > CLAR_TEST_OBJS += $(UNIT_TEST_DIR)/clar/clar.o > diff --git a/t/meson.build b/t/meson.build > index 1cf1d95f70..e224df58c8 100644 > --- a/t/meson.build > +++ b/t/meson.build > @@ -27,6 +27,7 @@ clar_test_suites = [ > 'unit-tests/u-trailer.c', > 'unit-tests/u-urlmatch-normalization.c', > 'unit-tests/u-utf8-width.c', > + 'unit-tests/u-utf8-ansi.c', > ] > > clar_sources = [ > diff --git a/t/unit-tests/u-utf8-ansi.c b/t/unit-tests/u-utf8-ansi.c > new file mode 100644 > index 0000000000..c19d8bce78 > --- /dev/null > +++ b/t/unit-tests/u-utf8-ansi.c > @@ -0,0 +1,14 @@ > +#include "unit-test.h" > +#include "utf8.h" > +#include "strbuf.h" > + > +/* > + * Test len limited functions for ANSI SGR lengths. > + */ > +void test_utf8_ansi__sgr_len(void) > +{ > + cl_assert_equal_i(0, display_mode_esc_sequence_len_counted("\033[m", 2)); > + cl_assert_equal_i(3, display_mode_esc_sequence_len_counted("\033[m", 3)); > + cl_assert_equal_i(5, display_mode_esc_sequence_len_counted("\033[31m", 5)); > + cl_assert_equal_i(7, display_mode_esc_sequence_len_counted("\033[0;31m", 7)); > +} Thanks. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted() 2026-11-17 21:43 ` Junio C Hamano @ 2026-11-17 22:03 ` Dirk Gouders 2026-11-18 1:33 ` Junio C Hamano 0 siblings, 1 reply; 11+ messages in thread From: Dirk Gouders @ 2026-11-17 22:03 UTC (permalink / raw) To: Junio C Hamano; +Cc: git On Tue, Nov 17 2026, Junio C Hamano wrote: > Dirk Gouders <dirk@gouders.net> writes: > >> You have better knowledge of the users of this function: would an early >> return perhaps handle a significant amount of cases? >> >> /* The shortest SGR sequence is "\033[m". */ >> if (len < 3) >> return 0; > > Do you have any benchmarks? I don't. > > I find it of dubious performance merit to run this comparison for a > long string as it is eaten one UTF-8 characters at a time by the > caller, and when the length is actually shorter than 3, then > runnning the remainder of the function for a few steps should not be > 10x more expensive than the above comparison, no? Meanwhile, I thought: even if the test would be a good idea, the "\033"-test probably is the one that handles most cases. Maybe I will look closer, but not now. Correctness is more important and I need to work on v2. >> And, why not add some tests (the untested function I showed had >> an off-by-one error): > > Sounds good. Should I just steal what you sent out? That wouldn't be stealing, of course you can use it. Regards, Dirk >> diff --git a/Makefile b/Makefile >> index 4c84eea6e2..3c8dc9ca40 100644 >> --- a/Makefile >> +++ b/Makefile >> @@ -1564,6 +1564,7 @@ CLAR_TEST_SUITES += u-strvec >> CLAR_TEST_SUITES += u-trailer >> CLAR_TEST_SUITES += u-urlmatch-normalization >> CLAR_TEST_SUITES += u-utf8-width >> +CLAR_TEST_SUITES += u-utf8-ansi >> CLAR_TEST_PROG = $(UNIT_TEST_BIN)/unit-tests$(X) >> CLAR_TEST_OBJS = $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(CLAR_TEST_SUITES)) >> CLAR_TEST_OBJS += $(UNIT_TEST_DIR)/clar/clar.o >> diff --git a/t/meson.build b/t/meson.build >> index 1cf1d95f70..e224df58c8 100644 >> --- a/t/meson.build >> +++ b/t/meson.build >> @@ -27,6 +27,7 @@ clar_test_suites = [ >> 'unit-tests/u-trailer.c', >> 'unit-tests/u-urlmatch-normalization.c', >> 'unit-tests/u-utf8-width.c', >> + 'unit-tests/u-utf8-ansi.c', >> ] >> >> clar_sources = [ >> diff --git a/t/unit-tests/u-utf8-ansi.c b/t/unit-tests/u-utf8-ansi.c >> new file mode 100644 >> index 0000000000..c19d8bce78 >> --- /dev/null >> +++ b/t/unit-tests/u-utf8-ansi.c >> @@ -0,0 +1,14 @@ >> +#include "unit-test.h" >> +#include "utf8.h" >> +#include "strbuf.h" >> + >> +/* >> + * Test len limited functions for ANSI SGR lengths. >> + */ >> +void test_utf8_ansi__sgr_len(void) >> +{ >> + cl_assert_equal_i(0, display_mode_esc_sequence_len_counted("\033[m", 2)); >> + cl_assert_equal_i(3, display_mode_esc_sequence_len_counted("\033[m", 3)); >> + cl_assert_equal_i(5, display_mode_esc_sequence_len_counted("\033[31m", 5)); >> + cl_assert_equal_i(7, display_mode_esc_sequence_len_counted("\033[0;31m", 7)); >> +} > > Thanks. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted() 2026-11-17 22:03 ` Dirk Gouders @ 2026-11-18 1:33 ` Junio C Hamano 2026-11-18 7:37 ` Dirk Gouders 0 siblings, 1 reply; 11+ messages in thread From: Junio C Hamano @ 2026-11-18 1:33 UTC (permalink / raw) To: Dirk Gouders; +Cc: git Dirk Gouders <dirk@gouders.net> writes: > Meanwhile, I thought: even if the test would be a good idea, the > "\033"-test probably is the one that handles most cases. > > Maybe I will look closer, but not now. Correctness is more important > and I need to work on v2. It would be nice if you can take this patch over and make it your [PATCH v2 1/2]. Thanks. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted() 2026-11-18 1:33 ` Junio C Hamano @ 2026-11-18 7:37 ` Dirk Gouders 0 siblings, 0 replies; 11+ messages in thread From: Dirk Gouders @ 2026-11-18 7:37 UTC (permalink / raw) To: Junio C Hamano; +Cc: git On Tue, Nov 17 2026, Junio C Hamano wrote: > Dirk Gouders <dirk@gouders.net> writes: > >> Meanwhile, I thought: even if the test would be a good idea, the >> "\033"-test probably is the one that handles most cases. >> >> Maybe I will look closer, but not now. Correctness is more important >> and I need to work on v2. > > It would be nice if you can take this patch over and make it your > [PATCH v2 1/2]. OK, I will do so, thank you. Regards, Dirk ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted() 2026-11-17 20:06 ` [PATCH v2] utf8: add display_mode_esc_sequence_len_counted() Junio C Hamano 2026-11-17 21:26 ` Dirk Gouders @ 2026-11-17 23:08 ` Andy Koppe 2026-11-17 23:25 ` Junio C Hamano 1 sibling, 1 reply; 11+ messages in thread From: Andy Koppe @ 2026-11-17 23:08 UTC (permalink / raw) To: Junio C Hamano; +Cc: git, Dirk Gouders Junio C Hamano wrote: > - while (isdigit(*p) || *p == ';') > + while (p < end && (isdigit(*p) || *p == ';')) While you're there, could you add colons (':') to that? ECMA-48 §5.4.2 defines them as sub-separators within the semicolon-separated parameters of control sequences. Colons are referred to as "3/10" there, aka ASCII 0x3A: https://ecma-international.org/wp-content/uploads/ECMA-48_5th_edition_june_1991.pdf#page=26 Maybe more practically relevant, xterm and its many more-or-less-compatible brethren accept colons in SGR sequences: "xterm allows either colons (standard) or semicolons (legacy) to separate the subparameters (but after the first colon, colons must be used)" https://invisible-island.net/xterm/ctlseqs/ctlseqs.html Andy ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted() 2026-11-17 23:08 ` Andy Koppe @ 2026-11-17 23:25 ` Junio C Hamano 2026-11-18 6:59 ` Andy Koppe 0 siblings, 1 reply; 11+ messages in thread From: Junio C Hamano @ 2026-11-17 23:25 UTC (permalink / raw) To: Andy Koppe; +Cc: git, Dirk Gouders Andy Koppe <andy.koppe@gmail.com> writes: > Junio C Hamano wrote: > >> - while (isdigit(*p) || *p == ';') >> + while (p < end && (isdigit(*p) || *p == ';')) > > While you're there, could you add colons (':') to that? Do we generate such a sequence that uses subseparators? IIRC, this code is not counting the display width of end-user supplied string. Rather, it gets string we paint in a buffer and then counts how much display column it occupies. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted() 2026-11-17 23:25 ` Junio C Hamano @ 2026-11-18 6:59 ` Andy Koppe 2026-11-18 7:43 ` Dirk Gouders 0 siblings, 1 reply; 11+ messages in thread From: Andy Koppe @ 2026-11-18 6:59 UTC (permalink / raw) To: Junio C Hamano; +Cc: git, Dirk Gouders Junio C Hamano wrote: > > Andy Koppe <andy.koppe@gmail.com> writes: > > > Junio C Hamano wrote: > > > >> - while (isdigit(*p) || *p == ';') > >> + while (p < end && (isdigit(*p) || *p == ';')) > > > > While you're there, could you add colons (':') to that? > > Do we generate such a sequence that uses subseparators? > > IIRC, this code is not counting the display width of end-user > supplied string. Rather, it gets string we paint in a buffer and > then counts how much display column it occupies. You're right that Git itself doesn't generate them (although future enhancements might change that). However, display_mode_esc_sequence_len() already operates on expanded custom formats and message text, e.g. in rewrap_message_tail() via strbuf_wrap(). And on stdin in git-column. They could all contain SGR sequences with colons, whose width currently isn't ignored as intended. (And my proposed make_line_colors_self_contained() wouldn't reset or replay them.) The check could be changed to `*p >= '0' && *p<=';'` to cheaply include colons. Andy ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] utf8: add display_mode_esc_sequence_len_counted() 2026-11-18 6:59 ` Andy Koppe @ 2026-11-18 7:43 ` Dirk Gouders 0 siblings, 0 replies; 11+ messages in thread From: Dirk Gouders @ 2026-11-18 7:43 UTC (permalink / raw) To: Andy Koppe; +Cc: Junio C Hamano, git On Wed, Nov 18 2026, Andy Koppe wrote: > Junio C Hamano wrote: >> >> Andy Koppe <andy.koppe@gmail.com> writes: >> >> > Junio C Hamano wrote: >> > >> >> - while (isdigit(*p) || *p == ';') >> >> + while (p < end && (isdigit(*p) || *p == ';')) >> > >> > While you're there, could you add colons (':') to that? >> >> Do we generate such a sequence that uses subseparators? >> >> IIRC, this code is not counting the display width of end-user >> supplied string. Rather, it gets string we paint in a buffer and >> then counts how much display column it occupies. > > You're right that Git itself doesn't generate them (although future > enhancements might change that). > > However, display_mode_esc_sequence_len() already operates on expanded > custom formats and message text, e.g. in rewrap_message_tail() via > strbuf_wrap(). And on stdin in git-column. They could all contain SGR > sequences with colons, whose width currently isn't ignored as > intended. (And my proposed make_line_colors_self_contained() wouldn't > reset or replay them.) > > The check could be changed to `*p >= '0' && *p<=';'` to cheaply include colons. I would suggest to not mix these changes simply for bisection purposes. And as far as I know, we would need an FSM to identify valid sequences if we are going to allow colons. Regards, Dirk ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2026-11-18 7:43 UTC | newest] Thread overview: 11+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-11-17 16:50 [PATCH] utf8: make display_mode_esc_sequence_len() take a length Junio C Hamano 2026-11-17 20:06 ` [PATCH v2] utf8: add display_mode_esc_sequence_len_counted() Junio C Hamano 2026-11-17 21:26 ` Dirk Gouders 2026-11-17 21:43 ` Junio C Hamano 2026-11-17 22:03 ` Dirk Gouders 2026-11-18 1:33 ` Junio C Hamano 2026-11-18 7:37 ` Dirk Gouders 2026-11-17 23:08 ` Andy Koppe 2026-11-17 23:25 ` Junio C Hamano 2026-11-18 6:59 ` Andy Koppe 2026-11-18 7:43 ` Dirk Gouders
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox