Re: [PATCH 2/3] meson: detect broken iconv that requires ICONV_RESTART_RESET

[Justin Tobler <jltobler@gmail.com>]

On 26/03/05 12:20PM, Patrick Steinhardt wrote:
> In d0cec08d70 (utf8.c: prepare workaround for iconv under macOS 14/15,
> 2026-01-12) we have introduced a new workaround for a broken version of
> libiconv on macOS. This workaround has for now only been wired up for
> our Makefile, so using Meson with such a broken version will fail.
> 
> We can rather easily detect the broken behaviour. Some encodings have
> different modes that can be switched to via an escape sequence. In the
> case of ISO-2022-JP this can be done via "<Esc>$B" and "<Esc>(J" to
> switch between ASCII and JIS modes. The bug now triggers when one does
> multiple calls to iconv(3p) to convert a string piece by piece, where
> the first call enters JIS mode. The second call forgets about the fact
> that it is still in JIS mode, and consequently it will incorrectly treat
> the input as ASCII, and thus the produced output is of course garbage.
> 
> Wire up a test that exercises this in Meson and, if it fails, set the
> `ICONV_RESTART_RESET` define.

Makes sense.

> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  meson.build | 26 ++++++++++++++++++++++++++
>  1 file changed, 26 insertions(+)
> 
> diff --git a/meson.build b/meson.build
> index ee3d9ced92..1b9be3d36c 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -1060,6 +1060,32 @@ if iconv.found()
>      ).returncode() != 0
>        libgit_c_args += '-DICONV_OMITS_BOM'
>      endif
> +
> +    if compiler.run('''
> +      #include <iconv.h>
> +      #include <string.h>
> +
> +      int main(int argc, const char *argv[])
> +      {
> +          char in[] = "\x1b\x24\x42\x24\x22\x24\x22\x1b\x28\x42", *inpos = in;

This appears to be the ISO-2022-JP encoded sequence that switches
between ASCII and JIS modes. Might we worth mentioning this in a
comment. :)

> +          char out[7] = { 0 }, *outpos = out;
> +          size_t insz = sizeof(in) - 1, outsz = 4;
> +          iconv_t conv = iconv_open("UTF-8", "ISO-2022-JP");
> +          if (!conv)
> +              return 1;
> +          if (iconv(conv, (void *) &inpos, &insz, &outpos, &outsz) != (size_t) -1)
> +              return 2;
> +          outsz = sizeof(out) - (outpos - out);
> +          if (iconv(conv, (void *) &inpos, &insz, &outpos, &outsz) == (size_t) -1)
> +              return 3;

Here we call iconv twice. If we have a bugged version of iconv, In the
second call we expect that it forgets the mode and returns garbage.

> +          return strcmp("\343\201\202\343\201\202", out) ? 4 : 0;

This appears to be the expected UTF-8 string. If the output doesn't
match, it indicates we are encountering this bug.

> +      }
> +    ''',
> +      dependencies: iconv,
> +      name: 'iconv handles restarts properly',
> +    ).returncode() != 0

Should we be more specific and expect the return code to be 4? It is my
understanding that these other scenarios would not be indicative of the
bug.

> +      libgit_c_args += '-DICONV_RESTART_RESET'
> +    endif
>    endif
>  else
>    libgit_c_args += '-DNO_ICONV'

The check here looks good, but without the surrounding context, it is
not immediately obvious to me what we are trying to do. It would
probably be nice to leave some breadcrumbs for future readers.

-Justin