t3900 failure on macOS, iconv(3) broken?

t3900 failure on macOS, iconv(3) broken?

[René Scharfe]

Hi all,

three tests of t3900 fail on macOS 26.1 for me:

  not ok 17 - ISO-2022-JP should be shown in UTF-8 now
  not ok 25 - ISO-2022-JP should be shown in UTF-8 now
  not ok 38 - commit --fixup into ISO-2022-JP from UTF-8

Here's the verbose output of the first one:

----- snip! -----
expecting success of 3900.17 'ISO-2022-JP should be shown in UTF-8 now':
                compare_with ISO-2022-JP "$TEST_DIRECTORY"/t3900/2-UTF-8.txt

--- /Users/x/src/git/t/t3900/2-UTF-8.txt 2024-10-01 19:43:24.605230684 +0000
+++ current     2025-12-08 21:52:45.786161909 +0000
@@ -1,4 +1,4 @@
 はれひほふ

 しているのが、いるので。
-濱浜ほれぷりぽれまびぐりろへ。
+濱浜ほれぷりぽれまび$0$j$m$X!#
not ok 17 - ISO-2022-JP should be shown in UTF-8 now
#
#                       compare_with ISO-2022-JP "$TEST_DIRECTORY"/t3900/2-UTF-8.txt
#
1..17
----- snap! -----

compare_with runs git show to display a commit message, which in this
case here was encoded using ISO-2022-JP and is supposed to be reencoded
to UTF-8, but git show only does that half-way -- the "$0$j$m$X!#" part
is from the original ISO-2022-JP representation.

That botched conversion is done by utf8.c::reencode_string_iconv().  It
calls iconv(3) to do the actual work, initially with an output buffer of
the same size as the input.  If the output needs more space the function
enlarges the buffer and calls iconv(3) again.

iconv(3) won't tell us how much space it needs, but it will report what
part it already managed to convert, so we can increase the buffer and
continue from there.  ISO-2022-JP has escape codes for switching between
character sets, so it's a stateful encoding.  I guess the iconv(3) on my
machine forgets the state at the end of part one and then messes up part
two.

I only noticed now because I used to compile with NO_ICONV for some
reason.

Is anyone else seeing this breakage as well?

Here's a patch that adds make variable ICONV_BREAKS.  It avoids the
breakage when enabled, by starting over again instead of continuing.

René


---
 Makefile |  6 ++++++
 utf8.c   | 13 +++++++++++++
 2 files changed, 19 insertions(+)

diff --git a/Makefile b/Makefile
index 6fc322ff88..cf8a0d3ee9 100644
--- a/Makefile
+++ b/Makefile
@@ -181,6 +181,9 @@ include shared.mak
 # byte-order mark (BOM) when writing UTF-16 or UTF-32 and always writes in
 # big-endian format.
 #
+# Define ICONV_BREAKS if your iconv implementation cannot reliably
+# break a string into valid substrings.
+#
 # Define NO_DEFLATE_BOUND if your zlib does not have deflateBound. Define
 # ZLIB_NG if you want to use zlib-ng instead of zlib.
 #
@@ -1836,6 +1839,9 @@ endif
 ifdef ICONV_OMITS_BOM
 	BASIC_CFLAGS += -DICONV_OMITS_BOM
 endif
+ifdef ICONV_BREAKS
+	BASIC_CFLAGS += -DICONV_BREAKS
+endif
 ifdef NEEDS_LIBGEN
 	EXTLIBS += -lgen
 endif
diff --git a/utf8.c b/utf8.c
index 35a0251939..ff0c541fbc 100644
--- a/utf8.c
+++ b/utf8.c
@@ -515,6 +515,19 @@ char *reencode_string_iconv(const char *in, size_t insz, iconv_t conv,
 			out = xrealloc(out, outalloc);
 			outpos = out + sofar;
 			outsz = outalloc - sofar - 1;
+#ifdef ICONV_BREAKS
+			/*
+			 * If iconv(3) messes up piecemeal conversions
+			 * then restore the original pointers, sizes,
+			 * and converter state, then retry converting
+			 * the full string using the reallocated buffer.
+			 */
+			insz += (char *)cp - in;
+			cp = (iconv_ibp)in;
+			outpos = out + bom_len;
+			outsz = outalloc - bom_len - 1;
+			iconv(conv, NULL, NULL, NULL, NULL);
+#endif
 		}
 		else {
 			*outpos = '\0';
-- 
2.52.0

Re: t3900 failure on macOS, iconv(3) broken?

[Koji Nakamaru]

On Tue, Dec 9, 2025 at 7:59 AM René Scharfe <l.s.r@web.de> wrote:
>
> Hi all,
>
> three tests of t3900 fail on macOS 26.1 for me:
>
>   not ok 17 - ISO-2022-JP should be shown in UTF-8 now
>   not ok 25 - ISO-2022-JP should be shown in UTF-8 now
>   not ok 38 - commit --fixup into ISO-2022-JP from UTF-8
>
> Here's the verbose output of the first one:
>
> ----- snip! -----
> expecting success of 3900.17 'ISO-2022-JP should be shown in UTF-8 now':
>                 compare_with ISO-2022-JP "$TEST_DIRECTORY"/t3900/2-UTF-8.txt
>
> --- /Users/x/src/git/t/t3900/2-UTF-8.txt 2024-10-01 19:43:24.605230684 +0000
> +++ current     2025-12-08 21:52:45.786161909 +0000
> @@ -1,4 +1,4 @@
>  はれひほふ
>
>  しているのが、いるので。
> -濱浜ほれぷりぽれまびぐりろへ。
> +濱浜ほれぷりぽれまび$0$j$m$X!#
> not ok 17 - ISO-2022-JP should be shown in UTF-8 now
> #
> #                       compare_with ISO-2022-JP "$TEST_DIRECTORY"/t3900/2-UTF-8.txt
> #
> 1..17
> ----- snap! -----
>
> compare_with runs git show to display a commit message, which in this
> case here was encoded using ISO-2022-JP and is supposed to be reencoded
> to UTF-8, but git show only does that half-way -- the "$0$j$m$X!#" part
> is from the original ISO-2022-JP representation.
>
> That botched conversion is done by utf8.c::reencode_string_iconv().  It
> calls iconv(3) to do the actual work, initially with an output buffer of
> the same size as the input.  If the output needs more space the function
> enlarges the buffer and calls iconv(3) again.
>
> iconv(3) won't tell us how much space it needs, but it will report what
> part it already managed to convert, so we can increase the buffer and
> continue from there.  ISO-2022-JP has escape codes for switching between
> character sets, so it's a stateful encoding.  I guess the iconv(3) on my
> machine forgets the state at the end of part one and then messes up part
> two.
>
> I only noticed now because I used to compile with NO_ICONV for some
> reason.
>
> Is anyone else seeing this breakage as well?
>
> Here's a patch that adds make variable ICONV_BREAKS.  It avoids the
> breakage when enabled, by starting over again instead of continuing.
>
> René
>
>
> ---
>  Makefile |  6 ++++++
>  utf8.c   | 13 +++++++++++++
>  2 files changed, 19 insertions(+)
>
> diff --git a/Makefile b/Makefile
> index 6fc322ff88..cf8a0d3ee9 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -181,6 +181,9 @@ include shared.mak
>  # byte-order mark (BOM) when writing UTF-16 or UTF-32 and always writes in
>  # big-endian format.
>  #
> +# Define ICONV_BREAKS if your iconv implementation cannot reliably
> +# break a string into valid substrings.
> +#
>  # Define NO_DEFLATE_BOUND if your zlib does not have deflateBound. Define
>  # ZLIB_NG if you want to use zlib-ng instead of zlib.
>  #
> @@ -1836,6 +1839,9 @@ endif
>  ifdef ICONV_OMITS_BOM
>         BASIC_CFLAGS += -DICONV_OMITS_BOM
>  endif
> +ifdef ICONV_BREAKS
> +       BASIC_CFLAGS += -DICONV_BREAKS
> +endif
>  ifdef NEEDS_LIBGEN
>         EXTLIBS += -lgen
>  endif
> diff --git a/utf8.c b/utf8.c
> index 35a0251939..ff0c541fbc 100644
> --- a/utf8.c
> +++ b/utf8.c
> @@ -515,6 +515,19 @@ char *reencode_string_iconv(const char *in, size_t insz, iconv_t conv,
>                         out = xrealloc(out, outalloc);
>                         outpos = out + sofar;
>                         outsz = outalloc - sofar - 1;
> +#ifdef ICONV_BREAKS
> +                       /*
> +                        * If iconv(3) messes up piecemeal conversions
> +                        * then restore the original pointers, sizes,
> +                        * and converter state, then retry converting
> +                        * the full string using the reallocated buffer.
> +                        */
> +                       insz += (char *)cp - in;
> +                       cp = (iconv_ibp)in;
> +                       outpos = out + bom_len;
> +                       outsz = outalloc - bom_len - 1;
> +                       iconv(conv, NULL, NULL, NULL, NULL);
> +#endif
>                 }
>                 else {
>                         *outpos = '\0';
> --
> 2.52.0

I checked a few cases:

* macOS 15.7.2:
  * These tests fail.
  * These tests pass if your ICONV_BREAKS patch is applied or
    ICONVDIR=/opt/homebrew/Cellar/libiconv/1.18 is specified.
* macOS 14.8.2
  * These tests pass.

It looks like the system iconv is broken on macOS 15 or later.

--
Koji Nakamaru

Re: t3900 failure on macOS, iconv(3) broken?

[Yee Cheng Chin]

> * macOS 14.8.2
>   * These tests pass.
> It looks like the system iconv is broken on macOS 15 or later.

I'm a little surprised that these tests pass in macOS 14 with native
(aka not from Homebrew) iconv. Apple replaced GNU iconv with a custom
version in macOS 14, which also caused a fair bit of breakages among
other third-party software. I would have expected this CI test to
break on macOS 14 unless this is a new behavior change / bug
introduced in macOS 15.

But yes, one way to fix it is to just provide the Homebrew GNU iconv
via ICONVDIR.

Re: t3900 failure on macOS, iconv(3) broken?

[Collin Funk]

Yee Cheng Chin <ychin.macvim@gmail.com> writes:

>> * macOS 14.8.2
>>   * These tests pass.
>> It looks like the system iconv is broken on macOS 15 or later.
>
> I'm a little surprised that these tests pass in macOS 14 with native
> (aka not from Homebrew) iconv. Apple replaced GNU iconv with a custom
> version in macOS 14, which also caused a fair bit of breakages among
> other third-party software. I would have expected this CI test to
> break on macOS 14 unless this is a new behavior change / bug
> introduced in macOS 15.
>
> But yes, one way to fix it is to just provide the Homebrew GNU iconv
> via ICONVDIR.

FWIW, the GNU iconv maintainer expressed frustration with the buggy
iconv implementation in macOS 14 [1]. He blamed Apple-specific patches
on FreeBSD's implementation.

Collin

[1] https://lists.gnu.org/archive/html/bug-gnulib/2024-05/msg00375.html

Re: t3900 failure on macOS, iconv(3) broken?

[Torsten Bögershausen]

On Mon, Dec 08, 2025 at 11:59:11PM +0100, René Scharfe wrote:
> Hi all,
> 
> three tests of t3900 fail on macOS 26.1 for me:
> 
>   not ok 17 - ISO-2022-JP should be shown in UTF-8 now
>   not ok 25 - ISO-2022-JP should be shown in UTF-8 now
>   not ok 38 - commit --fixup into ISO-2022-JP from UTF-8
> 
> Here's the verbose output of the first one:
> 
> ----- snip! -----
> expecting success of 3900.17 'ISO-2022-JP should be shown in UTF-8 now':
>                 compare_with ISO-2022-JP "$TEST_DIRECTORY"/t3900/2-UTF-8.txt
> 
> --- /Users/x/src/git/t/t3900/2-UTF-8.txt 2024-10-01 19:43:24.605230684 +0000
> +++ current     2025-12-08 21:52:45.786161909 +0000
> @@ -1,4 +1,4 @@
>  はれひほふ
> 
>  しているのが、いるので。
> -濱浜ほれぷりぽれまびぐりろへ。
> +濱浜ほれぷりぽれまび$0$j$m$X!#
> not ok 17 - ISO-2022-JP should be shown in UTF-8 now
> #
> #                       compare_with ISO-2022-JP "$TEST_DIRECTORY"/t3900/2-UTF-8.txt
> #
> 1..17
> ----- snap! -----
> 
> compare_with runs git show to display a commit message, which in this
> case here was encoded using ISO-2022-JP and is supposed to be reencoded
> to UTF-8, but git show only does that half-way -- the "$0$j$m$X!#" part
> is from the original ISO-2022-JP representation.
> 
> That botched conversion is done by utf8.c::reencode_string_iconv().  It
> calls iconv(3) to do the actual work, initially with an output buffer of
> the same size as the input.  If the output needs more space the function
> enlarges the buffer and calls iconv(3) again.
> 
> iconv(3) won't tell us how much space it needs, but it will report what
> part it already managed to convert, so we can increase the buffer and
> continue from there.  ISO-2022-JP has escape codes for switching between
> character sets, so it's a stateful encoding.  I guess the iconv(3) on my
> machine forgets the state at the end of part one and then messes up part
> two.
> 
> I only noticed now because I used to compile with NO_ICONV for some
> reason.
> 
> Is anyone else seeing this breakage as well?
> 
> Here's a patch that adds make variable ICONV_BREAKS.  It avoids the
> breakage when enabled, by starting over again instead of continuing.
> 
> René
> 
> 
> ---
>  Makefile |  6 ++++++
>  utf8.c   | 13 +++++++++++++
>  2 files changed, 19 insertions(+)
> 
> diff --git a/Makefile b/Makefile
> index 6fc322ff88..cf8a0d3ee9 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -181,6 +181,9 @@ include shared.mak
>  # byte-order mark (BOM) when writing UTF-16 or UTF-32 and always writes in
>  # big-endian format.
>  #
> +# Define ICONV_BREAKS if your iconv implementation cannot reliably
> +# break a string into valid substrings.
> +#
>  # Define NO_DEFLATE_BOUND if your zlib does not have deflateBound. Define
>  # ZLIB_NG if you want to use zlib-ng instead of zlib.
>  #
> @@ -1836,6 +1839,9 @@ endif
>  ifdef ICONV_OMITS_BOM
>  	BASIC_CFLAGS += -DICONV_OMITS_BOM
>  endif
> +ifdef ICONV_BREAKS
> +	BASIC_CFLAGS += -DICONV_BREAKS
> +endif
>  ifdef NEEDS_LIBGEN
>  	EXTLIBS += -lgen
>  endif
> diff --git a/utf8.c b/utf8.c
> index 35a0251939..ff0c541fbc 100644
> --- a/utf8.c
> +++ b/utf8.c
> @@ -515,6 +515,19 @@ char *reencode_string_iconv(const char *in, size_t insz, iconv_t conv,
>  			out = xrealloc(out, outalloc);
>  			outpos = out + sofar;
>  			outsz = outalloc - sofar - 1;
> +#ifdef ICONV_BREAKS
> +			/*
> +			 * If iconv(3) messes up piecemeal conversions
> +			 * then restore the original pointers, sizes,
> +			 * and converter state, then retry converting
> +			 * the full string using the reallocated buffer.
> +			 */
> +			insz += (char *)cp - in;
> +			cp = (iconv_ibp)in;
> +			outpos = out + bom_len;
> +			outsz = outalloc - bom_len - 1;
> +			iconv(conv, NULL, NULL, NULL, NULL);
> +#endif
>  		}
>  		else {
>  			*outpos = '\0';


I am not sure, if I understand the second call to iconv(NULL....)
Here is a slightly different patch.
Comments wellcome.


diff --git a/utf8.c b/utf8.c
index 35a0251939..b3c1dd2b59 100644
--- a/utf8.c
+++ b/utf8.c
@@ -486,10 +486,11 @@ int utf8_fprintf(FILE *stream, const char *format, ...)
 char *reencode_string_iconv(const char *in, size_t insz, iconv_t conv,
 			    size_t bom_len, size_t *outsz_p)
 {
-	size_t outsz, outalloc;
+	size_t outsz, outalloc, originsz;
 	char *out, *outpos;
 	iconv_ibp cp;
 
+	originsz = insz;
 	outsz = insz;
 	outalloc = st_add(outsz, 1 + bom_len); /* for terminating NUL */
 	out = xmalloc(outalloc);
@@ -515,6 +516,17 @@ char *reencode_string_iconv(const char *in, size_t insz, iconv_t conv,
 			out = xrealloc(out, outalloc);
 			outpos = out + sofar;
 			outsz = outalloc - sofar - 1;
+#ifdef __APPLE__
+			/*
+			 * Several version of iconv(3) mess up piecemeal conversions.
+			 * Restore the original pointers, sizes,
+			 * and converter state, then retry converting
+			 * the full string using the reallocated buffer.
+			 */
+                        insz = originsz;
+                        outpos = out + bom_len;
+                        cp = (iconv_ibp)in;
+#endif
 		}
 		else {
 			*outpos = '\0';

Re: t3900 failure on macOS, iconv(3) broken?

[René Scharfe]

On 12/9/25 5:33 PM, Torsten Bögershausen wrote:
> On Mon, Dec 08, 2025 at 11:59:11PM +0100, René Scharfe wrote:
>>
>> diff --git a/utf8.c b/utf8.c
>> index 35a0251939..ff0c541fbc 100644
>> --- a/utf8.c
>> +++ b/utf8.c
>> @@ -515,6 +515,19 @@ char *reencode_string_iconv(const char *in, size_t insz, iconv_t conv,
>>  			out = xrealloc(out, outalloc);
>>  			outpos = out + sofar;
>>  			outsz = outalloc - sofar - 1;
>> +#ifdef ICONV_BREAKS
>> +			/*
>> +			 * If iconv(3) messes up piecemeal conversions
>> +			 * then restore the original pointers, sizes,
>> +			 * and converter state, then retry converting
>> +			 * the full string using the reallocated buffer.
>> +			 */
>> +			insz += (char *)cp - in;
>> +			cp = (iconv_ibp)in;
>> +			outpos = out + bom_len;
>> +			outsz = outalloc - bom_len - 1;
>> +			iconv(conv, NULL, NULL, NULL, NULL);
>> +#endif
>>  		}
>>  		else {
>>  			*outpos = '\0';
> 
> 
> I am not sure, if I understand the second call to iconv(NULL....)

It resets the state of the converter, e.g. the current code page of
encodings that have multiple ones.

> Here is a slightly different patch.
> Comments wellcome.
> 
> 
> diff --git a/utf8.c b/utf8.c
> index 35a0251939..b3c1dd2b59 100644
> --- a/utf8.c
> +++ b/utf8.c
> @@ -486,10 +486,11 @@ int utf8_fprintf(FILE *stream, const char *format, ...)
>  char *reencode_string_iconv(const char *in, size_t insz, iconv_t conv,
>  			    size_t bom_len, size_t *outsz_p)
>  {
> -	size_t outsz, outalloc;
> +	size_t outsz, outalloc, originsz;
>  	char *out, *outpos;
>  	iconv_ibp cp;
>  
> +	originsz = insz;
>  	outsz = insz;
>  	outalloc = st_add(outsz, 1 + bom_len); /* for terminating NUL */
>  	out = xmalloc(outalloc);
> @@ -515,6 +516,17 @@ char *reencode_string_iconv(const char *in, size_t insz, iconv_t conv,
>  			out = xrealloc(out, outalloc);
>  			outpos = out + sofar;
>  			outsz = outalloc - sofar - 1;
> +#ifdef __APPLE__
> +			/*
> +			 * Several version of iconv(3) mess up piecemeal conversions.
> +			 * Restore the original pointers, sizes,
> +			 * and converter state, then retry converting
> +			 * the full string using the reallocated buffer.
> +			 */
> +                        insz = originsz;
> +                        outpos = out + bom_len;
> +                        cp = (iconv_ibp)in;

This forgets to reset outsz and the converter state.  With this patch
t0028-working-tree-encoding.sh seems to get stuck in an endless loop.

> +#endif
>  		}
>  		else {
>  			*outpos = '\0';

Re: t3900 failure on macOS, iconv(3) broken?

[Torsten Bögershausen]

On Tue, Dec 09, 2025 at 08:35:23PM +0100, René Scharfe wrote:
> On 12/9/25 5:33 PM, Torsten Bögershausen wrote:
> > On Mon, Dec 08, 2025 at 11:59:11PM +0100, René Scharfe wrote:
> >>
[snip]
> 
> This forgets to reset outsz and the converter state.  With this patch
> t0028-working-tree-encoding.sh seems to get stuck in an endless loop.

Thanks for testing.
I did another test here
(increase the outbuffer with only one byte per round, old MacOs)
and yes, we need to reset iconv.
Back to your patch. I think it is good to go further,
with one or 2 remarks, see TB
 
			out = xrealloc(out, outalloc);
			// TB: move into else outpos = out + sofar;
			// TB: move into else outsz = outalloc - sofar - 1;
// TB: We have seen different breakages of apple iconv. Should we run the same code
// on all versions of MacOs to be more future proof ?
// and do we need a Makefile knob, if one, and only one platform is affected ?
// I don't know
#ifdef __APPLE__
or
#ifdef ICONV_BREAKS
			/*
			 * If iconv(3) messes up piecemeal conversions
			 * then restore the original pointers, sizes,
			 * and converter state, then retry converting
			 * the full string using the reallocated buffer.
			 */
			insz += (char *)cp - in;    /* TB stumbled here: "in" is "const char *"
			                              And I didn't like the fact that insz is destroyed
						      and needs to be restored. That is why I had a originsz
						      (or szinorig ?)
			cp = (iconv_ibp)in;
			outpos = out + bom_len;
			outsz = outalloc - bom_len - 1;
			iconv(conv, NULL, NULL, NULL, NULL);
#else
			outpos = out + sofar;
			outsz = outalloc - sofar - 1;
#endif

Re: t3900 failure on macOS, iconv(3) broken?

[René Scharfe]

On 12/9/25 10:24 PM, Torsten Bögershausen wrote:
> On Tue, Dec 09, 2025 at 08:35:23PM +0100, René Scharfe wrote:
>> On 12/9/25 5:33 PM, Torsten Bögershausen wrote:
>>> On Mon, Dec 08, 2025 at 11:59:11PM +0100, René Scharfe wrote:
>>>>
> [snip]
>>
>> This forgets to reset outsz and the converter state.  With this patch
>> t0028-working-tree-encoding.sh seems to get stuck in an endless loop.
> 
> Thanks for testing.
> I did another test here
> (increase the outbuffer with only one byte per round, old MacOs)
> and yes, we need to reset iconv.
> Back to your patch. I think it is good to go further,
> with one or 2 remarks, see TB
>  
> 			out = xrealloc(out, outalloc);
> 			// TB: move into else outpos = out + sofar;
> 			// TB: move into else outsz = outalloc - sofar - 1;
> // TB: We have seen different breakages of apple iconv. Should we run the same code
> // on all versions of MacOs to be more future proof ?
> // and do we need a Makefile knob, if one, and only one platform is affected ?
> // I don't know
> #ifdef __APPLE__
> or
> #ifdef ICONV_BREAKS

macOS 14.8.2 reportedly doesn't have this particular issue, and I can
only hope that Apple will eventually fix that bug, so __APPLE__ seems a
bit too broad.

I'm also not thrilled about adding yet another build flag.  The patch I
just posted sidesteps the issue by using the existing ICONVDIR setting
to use libiconv from Homebrew.  We do that for gettext already, so it
should be fine..

> 			/*
> 			 * If iconv(3) messes up piecemeal conversions
> 			 * then restore the original pointers, sizes,
> 			 * and converter state, then retry converting
> 			 * the full string using the reallocated buffer.
> 			 */
> 			insz += (char *)cp - in;    /* TB stumbled here: "in" is "const char *"

We can add the const qualifier, but it won't affect the pointer
arithmetic.  Perhaps casting to iconv_ibp would be more consistent?

> 			                              And I didn't like the fact that insz is destroyed
> 						      and needs to be restored. That is why I had a originsz
> 						      (or szinorig ?)

Sure, storing the original value would work, but is slightly more effort
than subtracting the progress made so far.  originsz would only be used
if ICONV_BREAKS is defined, you'd need to declare it conditionally,
adding yet more overhead.

> 			cp = (iconv_ibp)in;
> 			outpos = out + bom_len;
> 			outsz = outalloc - bom_len - 1;
> 			iconv(conv, NULL, NULL, NULL, NULL);
> #else
> 			outpos = out + sofar;
> 			outsz = outalloc - sofar - 1;

I'd like to keep buffer increase and rollback separate.  Perhaps
splitting out the output buffer adjustment is worth it, though?  Not
sure. *shrug*

diff --git a/utf8.c b/utf8.c
index 35a0251939..c99243a63b 100644
--- a/utf8.c
+++ b/utf8.c
@@ -513,6 +513,18 @@ char *reencode_string_iconv(const char *in, size_t insz, iconv_t conv,
 			sofar = outpos - out;
 			outalloc = st_add3(sofar, st_mult(insz, 2), 32);
 			out = xrealloc(out, outalloc);
+#ifdef ICONV_BREAKS
+			/*
+			 * If iconv(3) messes up piecemeal conversions
+			 * then restore the original pointers, sizes,
+			 * and converter state, then retry converting
+			 * the full string using the reallocated buffer.
+			 */
+			insz += cp - (iconv_ibp)in;
+			cp = (iconv_ibp)in;
+			sofar = bom_len;
+			iconv(conv, NULL, NULL, NULL, NULL);
+#endif
 			outpos = out + sofar;
 			outsz = outalloc - sofar - 1;
 		}

> #endif

[PATCH] config.mak.uname: use iconv from Homebrew on macOS

[René Scharfe]

The library function iconv(3) supplied with macOS versions 15.7.2
(Sequoia) and 26.1 (Tahoe) is unreliable when doing conversions from
ISO-2022-JP to UTF-8 in multiple steps; t3900 reports this breakage:

  not ok 17 - ISO-2022-JP should be shown in UTF-8 now
  not ok 25 - ISO-2022-JP should be shown in UTF-8 now
  not ok 38 - commit --fixup into ISO-2022-JP from UTF-8

As a workaround, use libiconv from Homebrew, if available.

Helped-by: Koji Nakamaru <koji.nakamaru@gree.net>
Signed-off-by: René Scharfe <l.s.r@web.de>
---
 config.mak.uname | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/config.mak.uname b/config.mak.uname
index 1691c6ae6e..1b305e38c6 100644
--- a/config.mak.uname
+++ b/config.mak.uname
@@ -182,6 +182,13 @@ ifeq ($(uname_S),Darwin)
         endif
         endif
 
+	ifeq ($(shell test -d /opt/homebrew/opt/libiconv/ && echo y),y)
+		ICONVDIR ?= /opt/homebrew/opt/libiconv
+	endif
+	ifeq ($(shell test -d /usr/local/opt/libiconv/ && echo y),y)
+		ICONVDIR ?= /usr/local/opt/libiconv
+	endif
+
 	BASIC_LDFLAGS += -framework CoreServices
 endif
 ifeq ($(uname_S),SunOS)
-- 
2.52.0

Re: [PATCH] config.mak.uname: use iconv from Homebrew on macOS

[Yee Cheng Chin]

> +       ifeq ($(shell test -d /usr/local/opt/libiconv/ && echo y),y)
> +               ICONVDIR ?= /usr/local/opt/libiconv
> +       endif

One thing to keep in mind is that x86-64 Homebrew (which is the one
that uses the /usr/local/ location) can be installed via Rosetta 2 on
Apple Silicon Macs for testing (I use it myself). In that case you
wouldn't really want to use the /usr/local/opt/libiconv location. It
would be a somewhat niche case (the user has to be using Apple Silicon
Mac, and somehow has Rosetta Homebrew libiconv installed but not
native Homebrew libiconv), but could happen.

Re: [PATCH] config.mak.uname: use iconv from Homebrew on macOS

[René Scharfe]

On 12/9/25 9:39 PM, Yee Cheng Chin wrote:
>> +       ifeq ($(shell test -d /usr/local/opt/libiconv/ && echo y),y)
>> +               ICONVDIR ?= /usr/local/opt/libiconv
>> +       endif
> 
> One thing to keep in mind is that x86-64 Homebrew (which is the one
> that uses the /usr/local/ location) can be installed via Rosetta 2 on
> Apple Silicon Macs for testing (I use it myself). In that case you
> wouldn't really want to use the /usr/local/opt/libiconv location. It
> would be a somewhat niche case (the user has to be using Apple Silicon
> Mac, and somehow has Rosetta Homebrew libiconv installed but not
> native Homebrew libiconv), but could happen.

If you have libiconv from both x86-64 and aarch64 Homebrew, the patch
will use the latter.  If you only have the one from x86-64, it will
try to use that.  For gettext we already do the same.

You can force using the slightly broken system iconv as before by
compiling by setting the make variable ICONVDIR manually, e.g. with

$ make ICONVDIR=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr

René

Re: [PATCH] config.mak.uname: use iconv from Homebrew on macOS

[Carlo Marcelo Arenas Belón]

On Tue, Dec 09, 2025 at 08:35:34PM -0800, René Scharfe wrote:
> The library function iconv(3) supplied with macOS versions 15.7.2
> (Sequoia) and 26.1 (Tahoe) is unreliable when doing conversions from
> ISO-2022-JP to UTF-8 in multiple steps; t3900 reports this breakage:
> 
>   not ok 17 - ISO-2022-JP should be shown in UTF-8 now
>   not ok 25 - ISO-2022-JP should be shown in UTF-8 now
>   not ok 38 - commit --fixup into ISO-2022-JP from UTF-8
> 
> As a workaround, use libiconv from Homebrew, if available.

While I think Homebrew libraries are usually better than the ones that
come with the system, there are reasons why you would prefer not linking
with them and therefore forcing Homebrew as a dependency of your binaries.

One particularly good reason is that if you are building a fat binary (
useful if you target recent macOS which still supports x86_64 but don't
want to distribute different versions per CPU type) then the system
library (even if broken) might be preferred.

Slightly off topic, but should another patch that adds a `NO_HOMEBREW`
Makefile flag similar to `NO_FINK` or `NO_APPLE_PORTS` be added to help
drive this?

Carlo

Re: [PATCH] config.mak.uname: use iconv from Homebrew on macOS

[René Scharfe]

On 12/10/25 12:17 PM, Carlo Marcelo Arenas Belón wrote:
> On Tue, Dec 09, 2025 at 08:35:34PM -0800, René Scharfe wrote:
>> The library function iconv(3) supplied with macOS versions 15.7.2
>> (Sequoia) and 26.1 (Tahoe) is unreliable when doing conversions from
>> ISO-2022-JP to UTF-8 in multiple steps; t3900 reports this breakage:
>>
>>   not ok 17 - ISO-2022-JP should be shown in UTF-8 now
>>   not ok 25 - ISO-2022-JP should be shown in UTF-8 now
>>   not ok 38 - commit --fixup into ISO-2022-JP from UTF-8
>>
>> As a workaround, use libiconv from Homebrew, if available.
> 
> While I think Homebrew libraries are usually better than the ones that
> come with the system, there are reasons why you would prefer not linking
> with them and therefore forcing Homebrew as a dependency of your binaries.

The patch doesn't force, it just changes the default.  You can overrule
it by setting ICONVDIR explicitly, e.g. this will use the system's
libiconv:

$ make ICONVDIR=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr

> One particularly good reason is that if you are building a fat binary (
> useful if you target recent macOS which still supports x86_64 but don't
> want to distribute different versions per CPU type) then the system
> library (even if broken) might be preferred.

How do you do that?  By calling clang(1) with -arch x86_64 and -arch
arm64 and using lipo(1) on the results?  Is this possible with the
current make files?

> Slightly off topic, but should another patch that adds a `NO_HOMEBREW`
> Makefile flag similar to `NO_FINK` or `NO_APPLE_PORTS` be added to help
> drive this?

Sounds like a it could be useful to someone.

I'm a bit puzzled that they are implemented in a Darwin section of
Makefile.  config.mak.uname would be a better place, no?

René

Re: [PATCH] config.mak.uname: use iconv from Homebrew on macOS

[Junio C Hamano]

René Scharfe <l.s.r@web.de> writes:

>> Slightly off topic, but should another patch that adds a `NO_HOMEBREW`
>> Makefile flag similar to `NO_FINK` or `NO_APPLE_PORTS` be added to help
>> drive this?
>
> Sounds like a it could be useful to someone.

Hmph, how?  When you personally use fink or homebrew or whatever,
but are building binaries for others?

I am looking at relevant parts of Makefile

# Define NO_FINK if you are building on Darwin/Mac OS X, have Fink
# installed in /sw, but don't want GIT to link against any libraries
# installed there.  If defined you may specify your own (or Fink's)
# include directories and library directories by defining CFLAGS
# and LDFLAGS appropriately.
#
# Define NO_DARWIN_PORTS if you are building on Darwin/Mac OS X,
# have DarwinPorts installed in /opt/local, but don't want GIT to
# link against any libraries installed there.  If defined you may
# specify your own (or DarwinPort's) include directories and
# library directories by defining CFLAGS and LDFLAGS appropriately.

and notice that /opt/local/ is mentioned for DarwinPorts.  The patch
that started this thread talks about defaulting ICONVDIR to that of
Homebrew if available, but the new code checks /opt/homebrew and
then /usr/local/ (and let it override it).  Should the log message
be talking about DarwinPorts as well?


    As a workaround, set the default libiconv location to
    /opt/homebrew when the user has one from Homebrew, or
    to /opt/local when the user has one from MacPorts.

or something along the line?

By the way, for macOS newbies (like me), I wonder if a patch like
the attached may help?

Thanks.


----- >8 -----
Subject: [PATCH] Makefile: help macOS novices by mentioning MacPorts

Since Aug 2006, the DarwinPorts project renamed themselves as
MacPorts.  Those who are not intimately familiar with the Opensource
ecosystem around macOS from olden days, the name DarwinPorts may not
ring a bell, even when they are using MacPorts.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git c/Makefile w/Makefile
index 7e0f77e298..be027218a5 100644
--- c/Makefile
+++ w/Makefile
@@ -95,7 +95,8 @@ include shared.mak
 # and LDFLAGS appropriately.
 #
 # Define NO_DARWIN_PORTS if you are building on Darwin/Mac OS X,
-# have DarwinPorts installed in /opt/local, but don't want GIT to
+# have DarwinPorts (which is an old name for MacPorts) installed
+# in /opt/local, but don't want GIT to
 # link against any libraries installed there.  If defined you may
 # specify your own (or DarwinPort's) include directories and
 # library directories by defining CFLAGS and LDFLAGS appropriately.


    

Re: [PATCH] config.mak.uname: use iconv from Homebrew on macOS

[Carlo Marcelo Arenas Belón]

On Thu, Dec 11, 2025 at 11:53:07AM -0800, Junio C Hamano wrote:
> René Scharfe <l.s.r@web.de> writes:
> 
> >> Slightly off topic, but should another patch that adds a `NO_HOMEBREW`
> >> Makefile flag similar to `NO_FINK` or `NO_APPLE_PORTS` be added to help
> >> drive this?
> >
> > Sounds like a it could be useful to someone.
> 
> Hmph, how?  When you personally use fink or homebrew or whatever,
> but are building binaries for others?

correct; I think microsoft's git keeps a patch to do something like
that for other dependencies already.

the OS (at least up to the point were they drop support for Intel)
has EVERYTHING compiled as a fat binary.

% file /bin/ls
/bin/ls: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e]
/bin/ls (for architecture x86_64):	Mach-O 64-bit executable x86_64
/bin/ls (for architecture arm64e):	Mach-O 64-bit executable arm64e

it gets even more interesting when you look at the older releases
that also include support for 32-bit Intel/ARM/PowerPC.

> I am looking at relevant parts of Makefile
> 
> # Define NO_FINK if you are building on Darwin/Mac OS X, have Fink
> # installed in /sw, but don't want GIT to link against any libraries
> # installed there.  If defined you may specify your own (or Fink's)
> # include directories and library directories by defining CFLAGS
> # and LDFLAGS appropriately.
> #
> # Define NO_DARWIN_PORTS if you are building on Darwin/Mac OS X,
> # have DarwinPorts installed in /opt/local, but don't want GIT to
> # link against any libraries installed there.  If defined you may
> # specify your own (or DarwinPort's) include directories and
> # library directories by defining CFLAGS and LDFLAGS appropriately.
> 
> and notice that /opt/local/ is mentioned for DarwinPorts.  The patch
> that started this thread talks about defaulting ICONVDIR to that of
> Homebrew if available, but the new code checks /opt/homebrew and
> then /usr/local/ (and let it override it).  Should the log message
> be talking about DarwinPorts as well?
> 
> 
>     As a workaround, set the default libiconv location to
>     /opt/homebrew when the user has one from Homebrew, or
>     to /opt/local when the user has one from MacPorts.
> 
> or something along the line?

Since the original patch was only meant to help with Homebrew it
might not be worth mentioning the OTHER package managers IMHO.

I am hoping also there might be someone else that might be using
PKGSRC (usually in NetBSD)  or even gentoo's portage for that.

But I agree with you that the way we assume Homebrew is THE user
package manager that the OS uses might be problematic long term.

> By the way, for macOS newbies (like me), I wonder if a patch like
> the attached may help?

Did I read that correctly and you had found yourself forced into
running macOS at least somewhere?

> ----- >8 -----
> Subject: [PATCH] Makefile: help macOS novices by mentioning MacPorts
> 
> Since Aug 2006, the DarwinPorts project renamed themselves as
> MacPorts.  Those who are not intimately familiar with the Opensource
> ecosystem around macOS from olden days, the name DarwinPorts may not
> ring a bell, even when they are using MacPorts.
> 
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>  Makefile | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git c/Makefile w/Makefile
> index 7e0f77e298..be027218a5 100644
> --- c/Makefile
> +++ w/Makefile
> @@ -95,7 +95,8 @@ include shared.mak
>  # and LDFLAGS appropriately.
>  #
>  # Define NO_DARWIN_PORTS if you are building on Darwin/Mac OS X,
> -# have DarwinPorts installed in /opt/local, but don't want GIT to
> +# have DarwinPorts (which is an old name for MacPorts) installed
> +# in /opt/local, but don't want GIT to
>  # link against any libraries installed there.  If defined you may
>  # specify your own (or DarwinPort's) include directories and
>  # library directories by defining CFLAGS and LDFLAGS appropriately.
> 

It took years, but I woukd be honoured to provide a:

Reviewed-by: Carlo Marcelo Arenas Belon <carenas@gmail.com>

Carlo

PS. Sorry about the mispelling of my own name, but had yet to figure
out how to configure UTF-8 correctly in my latest setup.

Re: [PATCH] config.mak.uname: use iconv from Homebrew on macOS

[Junio C Hamano]

Carlo Marcelo Arenas Belón <carenas@gmail.com> writes:

>> I am looking at relevant parts of Makefile
>> 
>> # Define NO_FINK if you are building on Darwin/Mac OS X, have Fink
>> # installed in /sw, but don't want GIT to link against any libraries
>> # installed there.  If defined you may specify your own (or Fink's)
>> # include directories and library directories by defining CFLAGS
>> # and LDFLAGS appropriately.
>> #
>> # Define NO_DARWIN_PORTS if you are building on Darwin/Mac OS X,
>> # have DarwinPorts installed in /opt/local, but don't want GIT to
>> # link against any libraries installed there.  If defined you may
>> # specify your own (or DarwinPort's) include directories and
>> # library directories by defining CFLAGS and LDFLAGS appropriately.
>> 
>> and notice that /opt/local/ is mentioned for DarwinPorts.  The patch
>> that started this thread talks about defaulting ICONVDIR to that of
>> Homebrew if available, but the new code checks /opt/homebrew and
>> then /usr/local/ (and let it override it).  Should the log message
>> be talking about DarwinPorts as well?
>> 
>>     As a workaround, set the default libiconv location to
>>     /opt/homebrew when the user has one from Homebrew, or
>>     to /opt/local when the user has one from MacPorts.
>> 
>> or something along the line?
>
> Since the original patch was only meant to help with Homebrew it
> might not be worth mentioning the OTHER package managers IMHO.

Meaing that the original patch should have included only
/opt/homebrew and we should drop the part about /opt/local?

Or do you mean Homebrew may use /opt/local instead of /opt/homebrew
and both parts of the original patch are needed to give coverage to
different Homebrew installations?

If the latter, perhaps we can say something in the proposed commit
log message to explain having both /opt/{homebrew,local}/ is
necessary (and why)?

>> By the way, for macOS newbies (like me), I wonder if a patch like
>> the attached may help?
>
> Did I read that correctly and you had found yourself forced into
> running macOS at least somewhere?

No, but I do look at CI output that includes macOS jobs every day.

Re: [PATCH] config.mak.uname: use iconv from Homebrew on macOS

[René Scharfe]

On 12/12/25 3:20 AM, Junio C Hamano wrote:
> Carlo Marcelo Arenas Belón <carenas@gmail.com> writes:
> 
>>> I am looking at relevant parts of Makefile
>>>
>>> # Define NO_FINK if you are building on Darwin/Mac OS X, have Fink
>>> # installed in /sw, but don't want GIT to link against any libraries
>>> # installed there.  If defined you may specify your own (or Fink's)
>>> # include directories and library directories by defining CFLAGS
>>> # and LDFLAGS appropriately.
>>> #
>>> # Define NO_DARWIN_PORTS if you are building on Darwin/Mac OS X,
>>> # have DarwinPorts installed in /opt/local, but don't want GIT to
>>> # link against any libraries installed there.  If defined you may
>>> # specify your own (or DarwinPort's) include directories and
>>> # library directories by defining CFLAGS and LDFLAGS appropriately.
>>>
>>> and notice that /opt/local/ is mentioned for DarwinPorts.  The patch
>>> that started this thread talks about defaulting ICONVDIR to that of
>>> Homebrew if available, but the new code checks /opt/homebrew and
>>> then /usr/local/ (and let it override it).  Should the log message
>>> be talking about DarwinPorts as well?
>>>
>>>     As a workaround, set the default libiconv location to
>>>     /opt/homebrew when the user has one from Homebrew, or
>>>     to /opt/local when the user has one from MacPorts.
>>>
>>> or something along the line?
>>
>> Since the original patch was only meant to help with Homebrew it
>> might not be worth mentioning the OTHER package managers IMHO.
> 
> Meaing that the original patch should have included only
> /opt/homebrew and we should drop the part about /opt/local?
> 
> Or do you mean Homebrew may use /opt/local instead of /opt/homebrew
> and both parts of the original patch are needed to give coverage to
> different Homebrew installations?
> 
> If the latter, perhaps we can say something in the proposed commit
> log message to explain having both /opt/{homebrew,local}/ is
> necessary (and why)?

Homebrew uses /opt/homebrew for Apple Silicon and /usr/local for macOS
Intel (https://docs.brew.sh/Installation).

MacPorts née DarwinPorts uses /opt/local
(https://trac.macports.org/wiki/FAQ#defaultprefix).

Fink uses /opt/sw
(https://www.finkproject.org/faq/general.php?phpLang=en#why-sw).

The patch tries both Homebrew directories, the newer Apple Silicon
one first.

René

Re: [PATCH] config.mak.uname: use iconv from Homebrew on macOS

[Carlo Marcelo Arenas Belón]

On Fri, Dec 12, 2025 at 10:16:02AM -0800, René Scharfe wrote:
> On 12/12/25 3:20 AM, Junio C Hamano wrote:
> > Carlo Marcelo Arenas Belón <carenas@gmail.com> writes:
> > 
> >>> I am looking at relevant parts of Makefile
> >>>
> >>> # Define NO_FINK if you are building on Darwin/Mac OS X, have Fink
> >>> # installed in /sw, but don't want GIT to link against any libraries
> >>> # installed there.  If defined you may specify your own (or Fink's)
> >>> # include directories and library directories by defining CFLAGS
> >>> # and LDFLAGS appropriately.
> >>> #
> >>> # Define NO_DARWIN_PORTS if you are building on Darwin/Mac OS X,
> >>> # have DarwinPorts installed in /opt/local, but don't want GIT to
> >>> # link against any libraries installed there.  If defined you may
> >>> # specify your own (or DarwinPort's) include directories and
> >>> # library directories by defining CFLAGS and LDFLAGS appropriately.
> >>>
> >>> and notice that /opt/local/ is mentioned for DarwinPorts.  The patch
> >>> that started this thread talks about defaulting ICONVDIR to that of
> >>> Homebrew if available, but the new code checks /opt/homebrew and
> >>> then /usr/local/ (and let it override it).  Should the log message
> >>> be talking about DarwinPorts as well?
> >>>
> >>>     As a workaround, set the default libiconv location to
> >>>     /opt/homebrew when the user has one from Homebrew, or
> >>>     to /opt/local when the user has one from MacPorts.
> >>>
> >>> or something along the line?
> >>
> >> Since the original patch was only meant to help with Homebrew it
> >> might not be worth mentioning the OTHER package managers IMHO.
> > 
> > Meaing that the original patch should have included only
> > /opt/homebrew and we should drop the part about /opt/local?
> > 
> > Or do you mean Homebrew may use /opt/local instead of /opt/homebrew
> > and both parts of the original patch are needed to give coverage to
> > different Homebrew installations?
> > 
> > If the latter, perhaps we can say something in the proposed commit
> > log message to explain having both /opt/{homebrew,local}/ is
> > necessary (and why)?
> 
> Homebrew uses /opt/homebrew for Apple Silicon and /usr/local for macOS
> Intel (https://docs.brew.sh/Installation).

not always; you can install it anywhere you want, and indeed you might
need to (like I do) when given access to a remote instance of macOS
that you have no root on.

as you mentioned too, these settings are in the wrong Makefile (mainly
because they predate the split and creation of config.mak.uname) but
also because they are TOO peculiar of a case to be inside the latter
and because changing that might break some setups.

FWIW the use of "user" package managers is not unique to macOS. all
other UNIX have them as well, but luckily they are far less popular
and their use is declining (ex: AIX and Solaris the main two that remain
once HPUX is sunset, and not counting NONSTOP which we support directly)

Carlo

Re* [PATCH] config.mak.uname: use iconv from Homebrew on macOS

[Junio C Hamano]

René Scharfe <l.s.r@web.de> writes:

>> If the latter, perhaps we can say something in the proposed commit
>> log message to explain having both /opt/{homebrew,local}/ is
>> necessary (and why)?
>
> Homebrew uses /opt/homebrew for Apple Silicon and /usr/local for macOS
> Intel (https://docs.brew.sh/Installation).

Yup, that would be perfect.  Concisely explains why /opt/homebrew
and /usr/local are used in the patch.

I presume these are the default for homebrew and users could change
them to what suit their needs, but even then, what you did in your
patch is good; we are merely setting the appropriate defaults, and
those who customize these paths know that they must customize these
paths not just when they install homebrew but when building other
software packages like ours.

> Fink uses /opt/sw
> (https://www.finkproject.org/faq/general.php?phpLang=en#why-sw).

Perhaps they have a symlink or something from /sw to /opt/sw, then,
as our Makefile only talks about /sw and /opt/sw

Thanks.

Re: [PATCH] config.mak.uname: use iconv from Homebrew on macOS

[René Scharfe]

On 12/12/25 2:04 PM, Junio C Hamano wrote:
> René Scharfe <l.s.r@web.de> writes:
> 
>> Fink uses /opt/sw
>> (https://www.finkproject.org/faq/general.php?phpLang=en#why-sw).
> 
> Perhaps they have a symlink or something from /sw to /opt/sw, then,
> as our Makefile only talks about /sw and /opt/sw
Hmm, they changed that in https://github.com/fink/fink/commit/db958e12bf
six years ago.  Apparently we didn't get the memo.

René

Re: [PATCH] config.mak.uname: use iconv from Homebrew on macOS

[Junio C Hamano]

René Scharfe <l.s.r@web.de> writes:

> On 12/12/25 2:04 PM, Junio C Hamano wrote:
>> René Scharfe <l.s.r@web.de> writes:
>> 
>>> Fink uses /opt/sw
>>> (https://www.finkproject.org/faq/general.php?phpLang=en#why-sw).
>> 
>> Perhaps they have a symlink or something from /sw to /opt/sw, then,
>> as our Makefile only talks about /sw and /opt/sw
> Hmm, they changed that in https://github.com/fink/fink/commit/db958e12bf
> six years ago.  Apparently we didn't get the memo.

And apparently at least to us upstream Git, they do not matter;
otherwise we would have heard from their users.

Re: [PATCH] config.mak.uname: use iconv from Homebrew on macOS

[Torsten Bögershausen]

On Tue, Dec 09, 2025 at 08:35:34PM +0100, René Scharfe wrote:
> The library function iconv(3) supplied with macOS versions 15.7.2
> (Sequoia) and 26.1 (Tahoe) is unreliable when doing conversions from
> ISO-2022-JP to UTF-8 in multiple steps; t3900 reports this breakage:
> 
>   not ok 17 - ISO-2022-JP should be shown in UTF-8 now
>   not ok 25 - ISO-2022-JP should be shown in UTF-8 now
>   not ok 38 - commit --fixup into ISO-2022-JP from UTF-8
> 
> As a workaround, use libiconv from Homebrew, if available.
> 
> Helped-by: Koji Nakamaru <koji.nakamaru@gree.net>
> Signed-off-by: René Scharfe <l.s.r@web.de>
> ---
>  config.mak.uname | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/config.mak.uname b/config.mak.uname
> index 1691c6ae6e..1b305e38c6 100644
> --- a/config.mak.uname
> +++ b/config.mak.uname
> @@ -182,6 +182,13 @@ ifeq ($(uname_S),Darwin)
>          endif
>          endif
>  
> +	ifeq ($(shell test -d /opt/homebrew/opt/libiconv/ && echo y),y)
> +		ICONVDIR ?= /opt/homebrew/opt/libiconv
> +	endif
> +	ifeq ($(shell test -d /usr/local/opt/libiconv/ && echo y),y)
> +		ICONVDIR ?= /usr/local/opt/libiconv
> +	endif
> +
>  	BASIC_LDFLAGS += -framework CoreServices
>  endif
>  ifeq ($(uname_S),SunOS)
> -- 
> 2.52.0
> 

(Probaly a stupid question:) Does libiconv from homebrew provide UTF-8-MAC ?
And does t3910 pass ?

I just realized that I am building against libiconv from mac ports,
since years.
Digging into the Makefile shows that we have a switch:
NO_DARWIN_PORTS
(and another one for FINK)
Does it make sense to have a switch here as well ?

Re: [PATCH] config.mak.uname: use iconv from Homebrew on macOS

[René Scharfe]

On 12/10/25 5:42 PM, Torsten Bögershausen wrote:
> 
> (Probaly a stupid question:) Does libiconv from homebrew provide UTF-8-MAC ?

It has.  Curiously it also has the alias UTF8-MAC, while it doesn't have
UTF8 (but of course it has UTF-8).

> And does t3910 pass ?

Yes, except for:

not ok 23 - handle existing decomposed filenames # TODO known breakage

> I just realized that I am building against libiconv from mac ports,
> since years.
> Digging into the Makefile shows that we have a switch:
> NO_DARWIN_PORTS
> (and another one for FINK)
> Does it make sense to have a switch here as well ?
Carlo brought this up in the other thread as well, will reply there.

René

Re: [PATCH] config.mak.uname: use iconv from Homebrew on macOS

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 735 bytes --]

On 2025-12-09 at 19:35:34, René Scharfe wrote:
> The library function iconv(3) supplied with macOS versions 15.7.2
> (Sequoia) and 26.1 (Tahoe) is unreliable when doing conversions from
> ISO-2022-JP to UTF-8 in multiple steps; t3900 reports this breakage:
> 
>   not ok 17 - ISO-2022-JP should be shown in UTF-8 now
>   not ok 25 - ISO-2022-JP should be shown in UTF-8 now
>   not ok 38 - commit --fixup into ISO-2022-JP from UTF-8
> 
> As a workaround, use libiconv from Homebrew, if available.

I like this solution, since it means when Apple ships their own Git
(which doesn't use Homebrew), they will be incentivized to fix the
problem since the test fails.
-- 
brian m. carlson (they/them)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [PATCH] config.mak.uname: use iconv from Homebrew on macOS

[Junio C Hamano]

"brian m. carlson" <sandals@crustytoothpaste.net> writes:

> On 2025-12-09 at 19:35:34, René Scharfe wrote:
>> The library function iconv(3) supplied with macOS versions 15.7.2
>> (Sequoia) and 26.1 (Tahoe) is unreliable when doing conversions from
>> ISO-2022-JP to UTF-8 in multiple steps; t3900 reports this breakage:
>> 
>>   not ok 17 - ISO-2022-JP should be shown in UTF-8 now
>>   not ok 25 - ISO-2022-JP should be shown in UTF-8 now
>>   not ok 38 - commit --fixup into ISO-2022-JP from UTF-8
>> 
>> As a workaround, use libiconv from Homebrew, if available.
>
> I like this solution, since it means when Apple ships their own Git
> (which doesn't use Homebrew), they will be incentivized to fix the
> problem since the test fails.

Well, their build without Homebrew would fail with or without this
patch, no?  It is a good thing either way ;-)

Will queue.  Thanks.

Re: [PATCH] config.mak.uname: use iconv from Homebrew on macOS

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> "brian m. carlson" <sandals@crustytoothpaste.net> writes:
>
>> On 2025-12-09 at 19:35:34, René Scharfe wrote:
>>> The library function iconv(3) supplied with macOS versions 15.7.2
>>> (Sequoia) and 26.1 (Tahoe) is unreliable when doing conversions from
>>> ISO-2022-JP to UTF-8 in multiple steps; t3900 reports this breakage:
>>> 
>>>   not ok 17 - ISO-2022-JP should be shown in UTF-8 now
>>>   not ok 25 - ISO-2022-JP should be shown in UTF-8 now
>>>   not ok 38 - commit --fixup into ISO-2022-JP from UTF-8
>>> 
>>> As a workaround, use libiconv from Homebrew, if available.
>>
>> I like this solution, since it means when Apple ships their own Git
>> (which doesn't use Homebrew), they will be incentivized to fix the
>> problem since the test fails.
>
> Well, their build without Homebrew would fail with or without this
> patch, no?  It is a good thing either way ;-)

Does anybody know if a purely vanilla installation of macOS, without
any third-party software collection like homebrewk, is supposed to
be even serviceable?  That is, if somebody at Apple builds a version
of Git that they ship themselves (they do, don't they?), can they
untar the latest tarball on a vanilla macOS box, type "make test",
and expect it to pass?

Are there folks in the audience, with stakes in having such a thing
working, whether working for Apple or not, listening?  Can you
perhaps help us to get to that point?  The effort would involve (1)
fixing bugs in their own system, like this iconv issue, (2) marking
some part of the expectation unachievable by sprinkling !macOS
prerequisite in our tests, if bugs in their system like iconv cannot
be fixed for some reason, and (3) once we get to the point of
passing all tests, have a CI job to make sure we will stay clean.

Or do folks in macOS ecosystem already do something similar but
outside this mailing list, and it is useless for me to attempt to
help them?

Re: [PATCH] config.mak.uname: use iconv from Homebrew on macOS

[René Scharfe]

On 12/11/25 10:59 AM, Junio C Hamano wrote:
> 
> Does anybody know if a purely vanilla installation of macOS, without
> any third-party software collection like homebrewk, is supposed to
> be even serviceable?  That is, if somebody at Apple builds a version
> of Git that they ship themselves (they do, don't they?), can they
> untar the latest tarball on a vanilla macOS box, type "make test",
> and expect it to pass?

It seems so.  https://opensource.apple.com/releases/ points to
https://github.com/apple-oss-distributions/Git.  The latest tag is close
to v2.50.1:

$ git diff --stat -w v2.50.1 Git-155:src/git ':(exclude)*.git*'
 Documentation/fsck-msgids.adoc                   | 12 ------------
 Makefile                                         |  1 +
 attr.c                                           | 11 +++++++++++
 builtin/help.c                                   |  3 +--
 config.c                                         | 13 +++++++++++++
 config.h                                         |  3 +++
 generate-python.sh                               |  2 ++
 git-mergetool--lib.sh                            |  6 ++++--
 git-svn.perl                                     | 30 ++++++++++++++++++++++++++++++
 http.c                                           |  2 ++
 perl/header_templates/runtime_prefix.template.pl | 25 +++++++++++++++++++++++++
 sha1collisiondetection                           |  1 -
 t/t4014-format-patch.sh                          |  3 +--
 t/test-lib.sh                                    |  3 +++
 usage.c                                          | 20 ++++++++++++++++++++
 15 files changed, 116 insertions(+), 19 deletions(-)

Their top-level Makefile sets NO_GETTEXT, NO_FINK and NO_DARWIN_PORTS.

René

Re: [PATCH] config.mak.uname: use iconv from Homebrew on macOS

[Junio C Hamano]

René Scharfe <l.s.r@web.de> writes:

> On 12/11/25 10:59 AM, Junio C Hamano wrote:
>> 
>> Does anybody know if a purely vanilla installation of macOS, without
>> any third-party software collection like homebrewk, is supposed to
>> be even serviceable?  That is, if somebody at Apple builds a version
>> of Git that they ship themselves (they do, don't they?), can they
>> untar the latest tarball on a vanilla macOS box, type "make test",
>> and expect it to pass?
>
> It seems so.  https://opensource.apple.com/releases/ points to
> https://github.com/apple-oss-distributions/Git.  The latest tag is close
> to v2.50.1:
>
> $ git diff --stat -w v2.50.1 Git-155:src/git ':(exclude)*.git*'
>  Documentation/fsck-msgids.adoc                   | 12 ------------
>  Makefile                                         |  1 +
>  attr.c                                           | 11 +++++++++++
>  builtin/help.c                                   |  3 +--
>  config.c                                         | 13 +++++++++++++
>  config.h                                         |  3 +++
>  generate-python.sh                               |  2 ++
>  git-mergetool--lib.sh                            |  6 ++++--
>  git-svn.perl                                     | 30 ++++++++++++++++++++++++++++++
>  http.c                                           |  2 ++
>  perl/header_templates/runtime_prefix.template.pl | 25 +++++++++++++++++++++++++
>  sha1collisiondetection                           |  1 -
>  t/t4014-format-patch.sh                          |  3 +--
>  t/test-lib.sh                                    |  3 +++
>  usage.c                                          | 20 ++++++++++++++++++++
>  15 files changed, 116 insertions(+), 19 deletions(-)
>
> Their top-level Makefile sets NO_GETTEXT, NO_FINK and NO_DARWIN_PORTS.

Thanks.

Re: t3900 failure on macOS, iconv(3) broken?

[René Scharfe]

On 12/8/25 11:59 PM, René Scharfe wrote:
> 
> I only noticed now because I used to compile with NO_ICONV for some
> reason.
Nope, NO_ICONV does not work on macOS because compat/precompose_utf8.c
references reencode_string_iconv() and there's no (easy) way to disable
PRECOMPOSE_UNICODE.  I actually used ICONVDIR before, had forgotten
about it, got confused about it after deleting my config.mak and my
backup copy didn't set ICONVDIR, either.  Odd.  Anyway, just wanted to
correct the false impression that compiling with NO_ICONV on macOS would
be possible without source changes.

René

[PATCH v2 1/2] Makefile: add NO_HOMEBREW

[René Scharfe]

Allow disabling the use of Homebrew on macOS, or Linux for that matter,
like we already do for other package sources, MacPorts and Fink in
particular.  This is useful for packagers, or anyone else who wants to
control dependencies.

Suggested-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
Suggested-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: René Scharfe <l.s.r@web.de>
---
 Makefile         | 17 +++++++++++++++++
 config.mak.uname | 11 +++++------
 2 files changed, 22 insertions(+), 6 deletions(-)

diff --git a/Makefile b/Makefile
index 6fc322ff88..dbd2760d18 100644
--- a/Makefile
+++ b/Makefile
@@ -100,6 +100,9 @@ include shared.mak
 # specify your own (or DarwinPort's) include directories and
 # library directories by defining CFLAGS and LDFLAGS appropriately.
 #
+# Define NO_HOMEBREW if you have Homebrew and don't want Git to link
+# against libraries installed by it.
+#
 # Define NO_APPLE_COMMON_CRYPTO if you are building on Darwin/Mac OS X
 # and do not want to use Apple's CommonCrypto library.  This allows you
 # to provide your own OpenSSL library, for example from MacPorts.
@@ -1692,6 +1695,20 @@ ifeq ($(uname_S),Darwin)
 	PTHREAD_LIBS =
 endif
 
+ifndef NO_HOMEBREW
+        ifdef HOMEBREW_PREFIX
+		BASIC_CFLAGS += -I$(HOMEBREW_PREFIX)/include
+		BASIC_LDFLAGS += -L$(HOMEBREW_PREFIX)/lib
+        endif
+        ifdef HOMEBREW_GETTEXT_PREFIX
+		BASIC_CFLAGS += -I$(HOMEBREW_GETTEXT_PREFIX)/include
+		BASIC_LDFLAGS += -L$(HOMEBREW_GETTEXT_PREFIX)/lib
+        endif
+        ifdef HOMEBREW_MSGFMT
+		MSGFMT = $(HOMEBREW_MSGFMT)
+        endif
+endif
+
 ifdef NO_LIBGEN_H
 	COMPAT_CFLAGS += -DNO_LIBGEN_H
 	COMPAT_OBJS += compat/basename.o
diff --git a/config.mak.uname b/config.mak.uname
index 1691c6ae6e..a6521575ee 100644
--- a/config.mak.uname
+++ b/config.mak.uname
@@ -153,10 +153,10 @@ ifeq ($(uname_S),Darwin)
 	# `brew link --force gettext`, should be obsolete as of
 	# https://github.com/Homebrew/homebrew-core/pull/53489
         ifeq ($(shell test -d /usr/local/opt/gettext/ && echo y),y)
-		BASIC_CFLAGS += -I/usr/local/include -I/usr/local/opt/gettext/include
-		BASIC_LDFLAGS += -L/usr/local/lib -L/usr/local/opt/gettext/lib
+		HOMEBREW_PREFIX = /usr/local
+		HOMEBREW_GETTEXT_PREFIX = /usr/local/opt/gettext
                 ifeq ($(shell test -x /usr/local/opt/gettext/bin/msgfmt && echo y),y)
-			MSGFMT = /usr/local/opt/gettext/bin/msgfmt
+			HOMEBREW_MSGFMT = /usr/local/opt/gettext/bin/msgfmt
                 endif
 	# On newer ARM-based machines the default installation path has changed to
 	# /opt/homebrew. Include it in our search paths so that the user does not
@@ -166,10 +166,9 @@ ifeq ($(uname_S),Darwin)
 	# add gettext. The issue was fixed more than three years ago by now, and at
 	# that point there haven't been any ARM-based Macs yet.
         else ifeq ($(shell test -d /opt/homebrew/ && echo y),y)
-		BASIC_CFLAGS += -I/opt/homebrew/include
-		BASIC_LDFLAGS += -L/opt/homebrew/lib
+		HOMEBREW_PREFIX = /opt/homebrew
                 ifeq ($(shell test -x /opt/homebrew/bin/msgfmt && echo y),y)
-			MSGFMT = /opt/homebrew/bin/msgfmt
+			HOMEBREW_MSGFMT = /opt/homebrew/bin/msgfmt
                 endif
         endif
 
-- 
2.52.0

Re: [PATCH v2 1/2] Makefile: add NO_HOMEBREW

[Torsten Bögershausen]

On Sat, Dec 13, 2025 at 07:42:38PM +0100, René Scharfe wrote:
> Allow disabling the use of Homebrew on macOS, or Linux for that matter,
> like we already do for other package sources, MacPorts and Fink in
> particular.  This is useful for packagers, or anyone else who wants to
> control dependencies.

Good.
> 
> Suggested-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
> Suggested-by: Torsten Bögershausen <tboegi@web.de>
> Signed-off-by: René Scharfe <l.s.r@web.de>
> ---
>  Makefile         | 17 +++++++++++++++++
>  config.mak.uname | 11 +++++------
>  2 files changed, 22 insertions(+), 6 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 6fc322ff88..dbd2760d18 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -100,6 +100,9 @@ include shared.mak
>  # specify your own (or DarwinPort's) include directories and
>  # library directories by defining CFLAGS and LDFLAGS appropriately.
>  #
> +# Define NO_HOMEBREW if you have Homebrew and don't want Git to link
> +# against libraries installed by it.
> +#
Good
>  # Define NO_APPLE_COMMON_CRYPTO if you are building on Darwin/Mac OS X
>  # and do not want to use Apple's CommonCrypto library.  This allows you
>  # to provide your own OpenSSL library, for example from MacPorts.
> @@ -1692,6 +1695,20 @@ ifeq ($(uname_S),Darwin)
>  	PTHREAD_LIBS =
>  endif
>  
> +ifndef NO_HOMEBREW
> +        ifdef HOMEBREW_PREFIX

Question from a homebrew newbie, kind of:
Where do the HOMEBREW_PREFIX (and other HOMEBREW...) come from,
and what do they do ?

Running
git grep HOMEBREW
gives
ci/install-dependencies.sh:     export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1

Whould it make sense to have a few words here as a comment ?

> +		BASIC_CFLAGS += -I$(HOMEBREW_PREFIX)/include
> +		BASIC_LDFLAGS += -L$(HOMEBREW_PREFIX)/lib
> +        endif
> +        ifdef HOMEBREW_GETTEXT_PREFIX
> +		BASIC_CFLAGS += -I$(HOMEBREW_GETTEXT_PREFIX)/include
> +		BASIC_LDFLAGS += -L$(HOMEBREW_GETTEXT_PREFIX)/lib
> +        endif
> +        ifdef HOMEBREW_MSGFMT
> +		MSGFMT = $(HOMEBREW_MSGFMT)
> +        endif
> +endif
> +
>  ifdef NO_LIBGEN_H
>  	COMPAT_CFLAGS += -DNO_LIBGEN_H
>  	COMPAT_OBJS += compat/basename.o
> diff --git a/config.mak.uname b/config.mak.uname
> index 1691c6ae6e..a6521575ee 100644
> --- a/config.mak.uname
> +++ b/config.mak.uname
> @@ -153,10 +153,10 @@ ifeq ($(uname_S),Darwin)
>  	# `brew link --force gettext`, should be obsolete as of
>  	# https://github.com/Homebrew/homebrew-core/pull/53489
>          ifeq ($(shell test -d /usr/local/opt/gettext/ && echo y),y)
> -		BASIC_CFLAGS += -I/usr/local/include -I/usr/local/opt/gettext/include
> -		BASIC_LDFLAGS += -L/usr/local/lib -L/usr/local/opt/gettext/lib
> +		HOMEBREW_PREFIX = /usr/local
> +		HOMEBREW_GETTEXT_PREFIX = /usr/local/opt/gettext
>                  ifeq ($(shell test -x /usr/local/opt/gettext/bin/msgfmt && echo y),y)
> -			MSGFMT = /usr/local/opt/gettext/bin/msgfmt
> +			HOMEBREW_MSGFMT = /usr/local/opt/gettext/bin/msgfmt
>                  endif
>  	# On newer ARM-based machines the default installation path has changed to
>  	# /opt/homebrew. Include it in our search paths so that the user does not
> @@ -166,10 +166,9 @@ ifeq ($(uname_S),Darwin)
>  	# add gettext. The issue was fixed more than three years ago by now, and at
>  	# that point there haven't been any ARM-based Macs yet.
>          else ifeq ($(shell test -d /opt/homebrew/ && echo y),y)
> -		BASIC_CFLAGS += -I/opt/homebrew/include
> -		BASIC_LDFLAGS += -L/opt/homebrew/lib
> +		HOMEBREW_PREFIX = /opt/homebrew
>                  ifeq ($(shell test -x /opt/homebrew/bin/msgfmt && echo y),y)
> -			MSGFMT = /opt/homebrew/bin/msgfmt
> +			HOMEBREW_MSGFMT = /opt/homebrew/bin/msgfmt
>                  endif
>          endif
>  
> -- 
> 2.52.0
> 

Re: [PATCH v2 1/2] Makefile: add NO_HOMEBREW

[Junio C Hamano]

Torsten Bögershausen <tboegi@web.de> writes:

> On Sat, Dec 13, 2025 at 07:42:38PM +0100, René Scharfe wrote:
>> Allow disabling the use of Homebrew on macOS, or Linux for that matter,
>> like we already do for other package sources, MacPorts and Fink in
>> particular.  This is useful for packagers, or anyone else who wants to
>> control dependencies.
>
> Good.
>> 
>> Suggested-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
>> Suggested-by: Torsten Bögershausen <tboegi@web.de>
>> Signed-off-by: René Scharfe <l.s.r@web.de>
>> ---
>>  Makefile         | 17 +++++++++++++++++
>>  config.mak.uname | 11 +++++------
>>  2 files changed, 22 insertions(+), 6 deletions(-)
>> 
>> diff --git a/Makefile b/Makefile
>> index 6fc322ff88..dbd2760d18 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -100,6 +100,9 @@ include shared.mak
>>  # specify your own (or DarwinPort's) include directories and
>>  # library directories by defining CFLAGS and LDFLAGS appropriately.
>>  #
>> +# Define NO_HOMEBREW if you have Homebrew and don't want Git to link
>> +# against libraries installed by it.
>> +#
> Good
>>  # Define NO_APPLE_COMMON_CRYPTO if you are building on Darwin/Mac OS X
>>  # and do not want to use Apple's CommonCrypto library.  This allows you
>>  # to provide your own OpenSSL library, for example from MacPorts.
>> @@ -1692,6 +1695,20 @@ ifeq ($(uname_S),Darwin)
>>  	PTHREAD_LIBS =
>>  endif
>>  
>> +ifndef NO_HOMEBREW
>> +        ifdef HOMEBREW_PREFIX
>
> Question from a homebrew newbie, kind of:
> Where do the HOMEBREW_PREFIX (and other HOMEBREW...) come from,
> and what do they do ?

I understand these are purely _our_ thing.  HOMEBREW_PREFIX and
HOMEBREW_GETTEXT_PREFIX are set in config.mak.uname (added in this
patch).  I presume that those who installed homebrew at non-default
location and want to use homebrew would not set NO_HOMEBREW and set
HOMEBREW_PREFIX to the location they installed their homebrew which
would be different from the default set in config.mak.uname.  Those
who have homebrew installed at default location.

> Running
> git grep HOMEBREW
> gives
> ci/install-dependencies.sh:     export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
>
> Whould it make sense to have a few words here as a comment ?

Yeah, like 

# Define HOMEBREW_PREFIX to point at an appropriate directory, iff
# you want to use homebrew installed at a non-standard location.
# /opt/homebrew on Apple Silicon macOS and at /usr/local on Intel
# macOS are the standard locations (and you do not have to define
# this variable yourself).

perhaps?  Similarly for other variables.

Re: [PATCH v2 1/2] Makefile: add NO_HOMEBREW

[Torsten Bögershausen]

On Sun, Dec 14, 2025 at 04:13:14PM +0900, Junio C Hamano wrote:
> Torsten Bögershausen <tboegi@web.de> writes:
> 
> > On Sat, Dec 13, 2025 at 07:42:38PM +0100, René Scharfe wrote:
> >> Allow disabling the use of Homebrew on macOS, or Linux for that matter,
> >> like we already do for other package sources, MacPorts and Fink in
> >> particular.  This is useful for packagers, or anyone else who wants to
> >> control dependencies.
> >
> > Good.
> >> 
> >> Suggested-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
> >> Suggested-by: Torsten Bögershausen <tboegi@web.de>
> >> Signed-off-by: René Scharfe <l.s.r@web.de>
> >> ---
> >>  Makefile         | 17 +++++++++++++++++
> >>  config.mak.uname | 11 +++++------
> >>  2 files changed, 22 insertions(+), 6 deletions(-)
> >> 
> >> diff --git a/Makefile b/Makefile
> >> index 6fc322ff88..dbd2760d18 100644
> >> --- a/Makefile
> >> +++ b/Makefile
> >> @@ -100,6 +100,9 @@ include shared.mak
> >>  # specify your own (or DarwinPort's) include directories and
> >>  # library directories by defining CFLAGS and LDFLAGS appropriately.
> >>  #
> >> +# Define NO_HOMEBREW if you have Homebrew and don't want Git to link
> >> +# against libraries installed by it.
> >> +#
> > Good
> >>  # Define NO_APPLE_COMMON_CRYPTO if you are building on Darwin/Mac OS X
> >>  # and do not want to use Apple's CommonCrypto library.  This allows you
> >>  # to provide your own OpenSSL library, for example from MacPorts.
> >> @@ -1692,6 +1695,20 @@ ifeq ($(uname_S),Darwin)
> >>  	PTHREAD_LIBS =
> >>  endif
> >>  
> >> +ifndef NO_HOMEBREW
> >> +        ifdef HOMEBREW_PREFIX
> >
> > Question from a homebrew newbie, kind of:
> > Where do the HOMEBREW_PREFIX (and other HOMEBREW...) come from,
> > and what do they do ?
> 
> I understand these are purely _our_ thing.  HOMEBREW_PREFIX and
> HOMEBREW_GETTEXT_PREFIX are set in config.mak.uname (added in this
> patch).  I presume that those who installed homebrew at non-default
> location and want to use homebrew would not set NO_HOMEBREW and set
> HOMEBREW_PREFIX to the location they installed their homebrew which
> would be different from the default set in config.mak.uname.  Those
> who have homebrew installed at default location.
> 
> > Running
> > git grep HOMEBREW
> > gives
> > ci/install-dependencies.sh:     export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
> >
> > Whould it make sense to have a few words here as a comment ?
> 
> Yeah, like 
> 
> # Define HOMEBREW_PREFIX to point at an appropriate directory, iff
> # you want to use homebrew installed at a non-standard location.
> # /opt/homebrew on Apple Silicon macOS and at /usr/local on Intel
> # macOS are the standard locations (and you do not have to define
> # this variable yourself).
> 
> perhaps?  Similarly for other variables.

The main question is still, where the HOMEBREW_XXX variables
are used ?
I see that we define them in config.mak.uname
...I understand these are purely _our_ thing
That is what I don't get. It seems as if these are used when
compiling under/with homebrew ?

Re: [PATCH v2 1/2] Makefile: add NO_HOMEBREW

[Junio C Hamano]

Torsten Bögershausen <tboegi@web.de> writes:

> The main question is still, where the HOMEBREW_XXX variables
> are used ?
> I see that we define them in config.mak.uname
> ...I understand these are purely _our_ thing
> That is what I don't get. It seems as if these are used when
> compiling under/with homebrew ?

In a part I did not quote from your message I was responding to, to
which you are responding to in the message I am responding to, there
is this gem.

>> +ifndef NO_HOMEBREW
>> +        ifdef HOMEBREW_PREFIX
>
>Question from a homebrew newbie, kind of:
>Where do the HOMEBREW_PREFIX (and other HOMEBREW...) come from,
>and what do they do ?
>
>Running
>git grep HOMEBREW
>gives
>ci/install-dependencies.sh:     export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
>
>Whould it make sense to have a few words here as a comment ?
>
>> +		BASIC_CFLAGS += -I$(HOMEBREW_PREFIX)/include
>> +		BASIC_LDFLAGS += -L$(HOMEBREW_PREFIX)/lib
>> +        endif
>> +        ifdef HOMEBREW_GETTEXT_PREFIX
>> +		BASIC_CFLAGS += -I$(HOMEBREW_GETTEXT_PREFIX)/include
>> +		BASIC_LDFLAGS += -L$(HOMEBREW_GETTEXT_PREFIX)/lib
>> +        endif
>> +        ifdef HOMEBREW_MSGFMT
>> +		MSGFMT = $(HOMEBREW_MSGFMT)
>> +        endif
>> +endif
>> +

So, unless NO_HOMEBREW is set, HOMEBREW_PREFIX can be set by the
builder, or from config.mak.uname (when homebrew is installed in the
default location), and is added to BASIC_CFLAGS etc., which is how
these are used, IIUC.

Re: [PATCH v2 1/2] Makefile: add NO_HOMEBREW

[René Scharfe]

On 12/14/25 8:13 AM, Junio C Hamano wrote:
> Torsten Bögershausen <tboegi@web.de> writes:
> 
>> On Sat, Dec 13, 2025 at 07:42:38PM +0100, René Scharfe wrote:
>>> Allow disabling the use of Homebrew on macOS, or Linux for that matter,
>>> like we already do for other package sources, MacPorts and Fink in
>>> particular.  This is useful for packagers, or anyone else who wants to
>>> control dependencies.
>>
>> Good.
>>>
>>> Suggested-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
>>> Suggested-by: Torsten Bögershausen <tboegi@web.de>
>>> Signed-off-by: René Scharfe <l.s.r@web.de>
>>> ---
>>>  Makefile         | 17 +++++++++++++++++
>>>  config.mak.uname | 11 +++++------
>>>  2 files changed, 22 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/Makefile b/Makefile
>>> index 6fc322ff88..dbd2760d18 100644
>>> --- a/Makefile
>>> +++ b/Makefile
>>> @@ -100,6 +100,9 @@ include shared.mak
>>>  # specify your own (or DarwinPort's) include directories and
>>>  # library directories by defining CFLAGS and LDFLAGS appropriately.
>>>  #
>>> +# Define NO_HOMEBREW if you have Homebrew and don't want Git to link
>>> +# against libraries installed by it.
>>> +#
>> Good
>>>  # Define NO_APPLE_COMMON_CRYPTO if you are building on Darwin/Mac OS X
>>>  # and do not want to use Apple's CommonCrypto library.  This allows you
>>>  # to provide your own OpenSSL library, for example from MacPorts.
>>> @@ -1692,6 +1695,20 @@ ifeq ($(uname_S),Darwin)
>>>  	PTHREAD_LIBS =
>>>  endif
>>>  
>>> +ifndef NO_HOMEBREW
>>> +        ifdef HOMEBREW_PREFIX
>>
>> Question from a homebrew newbie, kind of:
>> Where do the HOMEBREW_PREFIX (and other HOMEBREW...) come from,
>> and what do they do ?
> 
> I understand these are purely _our_ thing.  HOMEBREW_PREFIX and
> HOMEBREW_GETTEXT_PREFIX are set in config.mak.uname (added in this
> patch).

Right.

> I presume that those who installed homebrew at non-default
> location and want to use homebrew would not set NO_HOMEBREW and set
> HOMEBREW_PREFIX to the location they installed their homebrew which
> would be different from the default set in config.mak.uname.  Those
> who have homebrew installed at default location.
> 
>> Running
>> git grep HOMEBREW
>> gives
>> ci/install-dependencies.sh:     export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
>>
>> Whould it make sense to have a few words here as a comment ?
> 
> Yeah, like 
> 
> # Define HOMEBREW_PREFIX to point at an appropriate directory, iff
> # you want to use homebrew installed at a non-standard location.
> # /opt/homebrew on Apple Silicon macOS and at /usr/local on Intel
> # macOS are the standard locations (and you do not have to define
> # this variable yourself).
> 
> perhaps?  Similarly for other variables.

Sounds useful, but before this can become a documented feature it
deserves more research and refinement.  The current code uses what it
can find in an ad-hoc manner, and the patches just extend this behavior
to libiconv.  A user-settable HOMEBREW_PREFIX would require a more
principled approach, so that overriding it affects the search for
gettext and libiconv.

I guess that would look like this in config.mak.uname:

ifeq ($(uname_S),Darwin)
ifeq ($(uname_M),arm64)
	HOMEBREW_PREFIX = /opt/homebrew
else
	HOMEBREW_PREFIX = /usr/local
endif
	USE_HOMEBREW_GETTEXT = IfAvailable
	USE_HOMEBREW_MSGFMT = IfAvailable
	USE_HOMEBREW_LIBICONV = IfAvailable
endif

... and in Makefile:

ifndef NO_HOMEBREW
ifdef HOMEBREW_PREFIX
ifdef USE_HOMEBREW_GETTEXT
	# magic!
endif
ifdef USE_HOMEBREW_MSGFMT
	# more magic!
endif
ifdef USE_HOMEBREW_LIBICONV
ifeq ($(shell test -d $(HOMEBREW_PREFIX)/opt/libiconv && echo y),y)
	ICONVDIR ?= $(HOMEBREW_PREFIX)/opt/libiconv
endif
endif
endif

Perhaps the magic parts just need to check for the existence of
$(HOMEBREW_PREFIX)/opt/gettext and use that, but the current code is
more complicated for some reason.

René

Re: [PATCH v2 1/2] Makefile: add NO_HOMEBREW

[Junio C Hamano]

René Scharfe <l.s.r@web.de> writes:

> Sounds useful, but before this can become a documented feature it
> deserves more research and refinement.  The current code uses what it
> can find in an ad-hoc manner, and the patches just extend this behavior
> to libiconv.  A user-settable HOMEBREW_PREFIX would require a more
> principled approach, so that overriding it affects the search for
> gettext and libiconv.

Oh, that is so true (but the specifics in macOS details is a bit
beyond my depth :/).

> I guess that would look like this in config.mak.uname:
>
> ifeq ($(uname_S),Darwin)
> ifeq ($(uname_M),arm64)
> 	HOMEBREW_PREFIX = /opt/homebrew
> else
> 	HOMEBREW_PREFIX = /usr/local
> endif
> 	USE_HOMEBREW_GETTEXT = IfAvailable
> 	USE_HOMEBREW_MSGFMT = IfAvailable
> 	USE_HOMEBREW_LIBICONV = IfAvailable
> endif
>
> ... and in Makefile:
>
> ifndef NO_HOMEBREW
> ifdef HOMEBREW_PREFIX
> ifdef USE_HOMEBREW_GETTEXT
> 	# magic!
> endif
> ifdef USE_HOMEBREW_MSGFMT
> 	# more magic!
> endif
> ifdef USE_HOMEBREW_LIBICONV
> ifeq ($(shell test -d $(HOMEBREW_PREFIX)/opt/libiconv && echo y),y)
> 	ICONVDIR ?= $(HOMEBREW_PREFIX)/opt/libiconv
> endif
> endif
> endif
>
> Perhaps the magic parts just need to check for the existence of
> $(HOMEBREW_PREFIX)/opt/gettext and use that, but the current code is
> more complicated for some reason.
>
> René

Re: [PATCH v2 1/2] Makefile: add NO_HOMEBREW

[René Scharfe]

On 12/15/25 12:19 AM, Junio C Hamano wrote:
> René Scharfe <l.s.r@web.de> writes:
> 
>> Sounds useful, but before this can become a documented feature it
>> deserves more research and refinement.  The current code uses what it
>> can find in an ad-hoc manner, and the patches just extend this behavior
>> to libiconv.  A user-settable HOMEBREW_PREFIX would require a more
>> principled approach, so that overriding it affects the search for
>> gettext and libiconv.
> 
> Oh, that is so true (but the specifics in macOS details is a bit
> beyond my depth :/).
Similar for me, but how hard can it be? :-P.  v3 coming, but needs
thorough review and testing.

René

[PATCH v2 2/2] config.mak.uname: use iconv from Homebrew on macOS

[René Scharfe]

The library function iconv(3) supplied with macOS versions 15.7.2
(Sequoia) and 26.1 (Tahoe) is unreliable when doing conversions from
ISO-2022-JP to UTF-8 in multiple steps; t3900 reports this breakage:

  not ok 17 - ISO-2022-JP should be shown in UTF-8 now
  not ok 25 - ISO-2022-JP should be shown in UTF-8 now
  not ok 38 - commit --fixup into ISO-2022-JP from UTF-8

As a workaround, use libiconv from Homebrew, if available.  Search it in
its default locations: /opt/homebrew for Apple Silicon and /usr/local
for macOS Intel, with the former taking precedence.  Respect ICONVDIR if
already set by the user, though.

Helped-by: Koji Nakamaru <koji.nakamaru@gree.net>
Signed-off-by: René Scharfe <l.s.r@web.de>
---
 Makefile         | 3 +++
 config.mak.uname | 7 +++++++
 2 files changed, 10 insertions(+)

diff --git a/Makefile b/Makefile
index dbd2760d18..80af832529 100644
--- a/Makefile
+++ b/Makefile
@@ -1707,6 +1707,9 @@ ifndef NO_HOMEBREW
         ifdef HOMEBREW_MSGFMT
 		MSGFMT = $(HOMEBREW_MSGFMT)
         endif
+        ifdef HOMEBREW_ICONVDIR
+		ICONVDIR ?= $(HOMEBREW_ICONVDIR)
+        endif
 endif
 
 ifdef NO_LIBGEN_H
diff --git a/config.mak.uname b/config.mak.uname
index a6521575ee..a926943141 100644
--- a/config.mak.uname
+++ b/config.mak.uname
@@ -172,6 +172,13 @@ ifeq ($(uname_S),Darwin)
                 endif
         endif
 
+        ifeq ($(shell test -d /usr/local/opt/libiconv/ && echo y),y)
+		HOMEBREW_ICONVDIR = /usr/local/opt/libiconv
+        endif
+        ifeq ($(shell test -d /opt/homebrew/opt/libiconv/ && echo y),y)
+		HOMEBREW_ICONVDIR = /opt/homebrew/opt/libiconv
+        endif
+
 	# The builtin FSMonitor on MacOS builds upon Simple-IPC.  Both require
 	# Unix domain sockets and PThreads.
         ifndef NO_PTHREADS
-- 
2.52.0

[PATCH v3 1/2] macOS: make Homebrew use configurable

[René Scharfe]

On macOS we opportunistically use Homebrew-installed versions of
gettext(3) and msgfmt(1).  Make that behavior configurable by providing
make variables to disable Homebrew usage (NO_HOMEBREW), to allow using a
non-default installation location (HOMEBREW_PREFIX), and to control the
use of the individual items (USE_HOMEBREW_GETTEXT, USE_HOMEBREW_MSGFMT).

Precisely Link the gettext keg (the opt/gettext subdirectory) instead of
risking to link random other Homebrew-installed libraries as well.

Signed-off-by: René Scharfe <l.s.r@web.de>
---
 Makefile         | 28 ++++++++++++++++++++++++++++
 config.mak.uname | 28 ++++++----------------------
 2 files changed, 34 insertions(+), 22 deletions(-)

diff --git a/Makefile b/Makefile
index cf3f4b585f..a97e9e4d7d 100644
--- a/Makefile
+++ b/Makefile
@@ -100,6 +100,18 @@ include shared.mak
 # specify your own (or DarwinPort's) include directories and
 # library directories by defining CFLAGS and LDFLAGS appropriately.
 #
+# Define NO_HOMEBREW if you don't want to use libraries and commands
+# installed by Homebrew.
+#
+# Define HOMEBREW_PREFIX if you have Homebrew installed in a non-default
+# location on macOS or on Linux and want to use it.
+#
+# Define USE_HOMEBREW_GETTEXT to link against the gettext library
+# installed by Homebrew, if present.
+#
+# Define USE_HOMEBREW_MSGFMT to use the msgfmt command installed by
+# Homebrew to compile message catalogs during build, if present.
+#
 # Define NO_APPLE_COMMON_CRYPTO if you are building on Darwin/Mac OS X
 # and do not want to use Apple's CommonCrypto library.  This allows you
 # to provide your own OpenSSL library, for example from MacPorts.
@@ -1692,6 +1704,22 @@ ifeq ($(uname_S),Darwin)
 	PTHREAD_LIBS =
 endif
 
+ifndef NO_HOMEBREW
+ifdef HOMEBREW_PREFIX
+ifdef USE_HOMEBREW_GETTEXT
+ifeq ($(shell test -d $(HOMEBREW_PREFIX)/opt/gettext && echo y),y)
+	BASIC_CFLAGS += -I$(HOMEBREW_PREFIX)/opt/gettext/include
+	BASIC_LDFLAGS += -L$(HOMEBREW_PREFIX)/opt/gettext/lib
+endif
+endif
+ifdef USE_HOMEBREW_MSGFMT
+ifeq ($(shell test -x $(HOMEBREW_PREFIX)/opt/gettext/msgfmt && echo y),y)
+	MSGFMT = $(HOMEBREW_PREFIX)/opt/gettext/msgfmt
+endif
+endif
+endif
+endif
+
 ifdef NO_LIBGEN_H
 	COMPAT_CFLAGS += -DNO_LIBGEN_H
 	COMPAT_OBJS += compat/basename.o
diff --git a/config.mak.uname b/config.mak.uname
index 1691c6ae6e..54e3a26649 100644
--- a/config.mak.uname
+++ b/config.mak.uname
@@ -149,29 +149,13 @@ ifeq ($(uname_S),Darwin)
 	CSPRNG_METHOD = arc4random
 	USE_ENHANCED_BASIC_REGULAR_EXPRESSIONS = YesPlease
 
-	# Workaround for `gettext` being keg-only and not even being linked via
-	# `brew link --force gettext`, should be obsolete as of
-	# https://github.com/Homebrew/homebrew-core/pull/53489
-        ifeq ($(shell test -d /usr/local/opt/gettext/ && echo y),y)
-		BASIC_CFLAGS += -I/usr/local/include -I/usr/local/opt/gettext/include
-		BASIC_LDFLAGS += -L/usr/local/lib -L/usr/local/opt/gettext/lib
-                ifeq ($(shell test -x /usr/local/opt/gettext/bin/msgfmt && echo y),y)
-			MSGFMT = /usr/local/opt/gettext/bin/msgfmt
-                endif
-	# On newer ARM-based machines the default installation path has changed to
-	# /opt/homebrew. Include it in our search paths so that the user does not
-	# have to configure this manually.
-	#
-	# Note that we do not employ the same workaround as above where we manually
-	# add gettext. The issue was fixed more than three years ago by now, and at
-	# that point there haven't been any ARM-based Macs yet.
-        else ifeq ($(shell test -d /opt/homebrew/ && echo y),y)
-		BASIC_CFLAGS += -I/opt/homebrew/include
-		BASIC_LDFLAGS += -L/opt/homebrew/lib
-                ifeq ($(shell test -x /opt/homebrew/bin/msgfmt && echo y),y)
-			MSGFMT = /opt/homebrew/bin/msgfmt
-                endif
+        ifeq ($(uname_M),arm64)
+		HOMEBREW_PREFIX = /opt/homebrew
+        else
+		HOMEBREW_PREFIX = /usr/local
         endif
+	USE_HOMEBREW_GETTEXT = YesPlease
+	USE_HOMEBREW_MSGFMT = YesPlease
 
 	# The builtin FSMonitor on MacOS builds upon Simple-IPC.  Both require
 	# Unix domain sockets and PThreads.
-- 
2.52.0

Re: [PATCH v3 1/2] macOS: make Homebrew use configurable

[René Scharfe]

On 12/16/25 7:53 PM, René Scharfe wrote:
> On macOS we opportunistically use Homebrew-installed versions of
> gettext(3) and msgfmt(1).  Make that behavior configurable by providing
> make variables to disable Homebrew usage (NO_HOMEBREW), to allow using a
> non-default installation location (HOMEBREW_PREFIX), and to control the
> use of the individual items (USE_HOMEBREW_GETTEXT, USE_HOMEBREW_MSGFMT).
> 
> Precisely Link the gettext keg (the opt/gettext subdirectory) instead of
> risking to link random other Homebrew-installed libraries as well.
> 
> Signed-off-by: René Scharfe <l.s.r@web.de>
> ---
>  Makefile         | 28 ++++++++++++++++++++++++++++
>  config.mak.uname | 28 ++++++----------------------
>  2 files changed, 34 insertions(+), 22 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index cf3f4b585f..a97e9e4d7d 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -100,6 +100,18 @@ include shared.mak
>  # specify your own (or DarwinPort's) include directories and
>  # library directories by defining CFLAGS and LDFLAGS appropriately.
>  #
> +# Define NO_HOMEBREW if you don't want to use libraries and commands
> +# installed by Homebrew.
> +#
> +# Define HOMEBREW_PREFIX if you have Homebrew installed in a non-default
> +# location on macOS or on Linux and want to use it.
> +#
> +# Define USE_HOMEBREW_GETTEXT to link against the gettext library
> +# installed by Homebrew, if present.
> +#
> +# Define USE_HOMEBREW_MSGFMT to use the msgfmt command installed by
> +# Homebrew to compile message catalogs during build, if present.

Do we even need these fine-grained USE_ variables?

René

Re: [PATCH v3 1/2] macOS: make Homebrew use configurable

[Torsten Bögershausen]

> > +#
> > +# Define USE_HOMEBREW_GETTEXT to link against the gettext library
> > +# installed by Homebrew, if present.
> > +#
> > +# Define USE_HOMEBREW_MSGFMT to use the msgfmt command installed by
> > +# Homebrew to compile message catalogs during build, if present.
> 
> Do we even need these fine-grained USE_ variables?

If someone asks me: Probably not.

> 
> René
> 
> 

[PATCH v3 2/2] macOS: use iconv from Homebrew if present

[René Scharfe]

The library function iconv(3) supplied with macOS versions 15.7.2
(Sequoia) and 26.1 (Tahoe) is unreliable when doing conversions from
ISO-2022-JP to UTF-8 in multiple steps; t3900 reports this breakage:

  not ok 17 - ISO-2022-JP should be shown in UTF-8 now
  not ok 25 - ISO-2022-JP should be shown in UTF-8 now
  not ok 38 - commit --fixup into ISO-2022-JP from UTF-8

As a workaround, use libiconv from Homebrew, if available.  Search it in
its default locations: /opt/homebrew for Apple Silicon and /usr/local
for macOS Intel, with the former taking precedence.  Respect ICONVDIR if
already set by the user, though.

Helped-by: Koji Nakamaru <koji.nakamaru@gree.net>
Signed-off-by: René Scharfe <l.s.r@web.de>
---
 Makefile         | 8 ++++++++
 config.mak.uname | 1 +
 2 files changed, 9 insertions(+)

diff --git a/Makefile b/Makefile
index a97e9e4d7d..307dac3c03 100644
--- a/Makefile
+++ b/Makefile
@@ -112,6 +112,9 @@ include shared.mak
 # Define USE_HOMEBREW_MSGFMT to use the msgfmt command installed by
 # Homebrew to compile message catalogs during build, if present.
 #
+# Define USE_HOMEBREW_LIBICONV to link against libiconv installed by
+# Homebrew, if present.
+#
 # Define NO_APPLE_COMMON_CRYPTO if you are building on Darwin/Mac OS X
 # and do not want to use Apple's CommonCrypto library.  This allows you
 # to provide your own OpenSSL library, for example from MacPorts.
@@ -1717,6 +1720,11 @@ ifeq ($(shell test -x $(HOMEBREW_PREFIX)/opt/gettext/msgfmt && echo y),y)
 	MSGFMT = $(HOMEBREW_PREFIX)/opt/gettext/msgfmt
 endif
 endif
+ifdef USE_HOMEBREW_LIBICONV
+ifeq ($(shell test -d $(HOMEBREW_PREFIX)/opt/libiconv && echo y),y)
+	ICONVDIR ?= $(HOMEBREW_PREFIX)/opt/libiconv
+endif
+endif
 endif
 endif
 
diff --git a/config.mak.uname b/config.mak.uname
index 54e3a26649..e2e93e9dc5 100644
--- a/config.mak.uname
+++ b/config.mak.uname
@@ -156,6 +156,7 @@ ifeq ($(uname_S),Darwin)
         endif
 	USE_HOMEBREW_GETTEXT = YesPlease
 	USE_HOMEBREW_MSGFMT = YesPlease
+	USE_HOMEBREW_LIBICONV = YesPlease
 
 	# The builtin FSMonitor on MacOS builds upon Simple-IPC.  Both require
 	# Unix domain sockets and PThreads.
-- 
2.52.0

[PATCH v4 0/2] macOS: use iconv from Homebrew if needed and present

[René Scharfe]

Changes since v3:
- Removed unnecessary USE_HOMEBREW_GETTEXT and USE_HOMEBREW_MSGFMT.
- Set USE_HOMEBREW_LIBICONV only on known-broken OS versions.

Changes since v2:
- Added HOMEBREW_PREFIX.
- Added USE_HOMEBREW_GETTEXT, USE_HOMEBREW_MSGFMT and USE_HOMEBREW_LIBICONV.

Changes since v1:
- Added NO_HOMEBREW.

  macOS: make Homebrew use configurable
  macOS: use iconv from Homebrew if needed and present

 Makefile         | 26 ++++++++++++++++++++++++++
 config.mak.uname | 30 ++++++++----------------------
 2 files changed, 34 insertions(+), 22 deletions(-)

-- 
2.52.0

[PATCH v4 1/2] macOS: make Homebrew use configurable

[René Scharfe]

On macOS we opportunistically use Homebrew-installed versions of
gettext(3) and msgfmt(1).  Make that behavior configurable by providing
make variables to disable Homebrew usage (NO_HOMEBREW) and to allow
using a non-default installation location (HOMEBREW_PREFIX).

Include and link only the gettext keg via the symlink opt/gettext
pointing to its installed version instead of using the Homebrew prefix.
This is simpler and prevents accidentally including other libraries.

Suggested-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
Suggested-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: René Scharfe <l.s.r@web.de>
---
 Makefile         | 18 ++++++++++++++++++
 config.mak.uname | 26 ++++----------------------
 2 files changed, 22 insertions(+), 22 deletions(-)

diff --git a/Makefile b/Makefile
index 89d8d73ec0..9aef22c032 100644
--- a/Makefile
+++ b/Makefile
@@ -101,6 +101,12 @@ include shared.mak
 # specify your own (or DarwinPort's) include directories and
 # library directories by defining CFLAGS and LDFLAGS appropriately.
 #
+# Define NO_HOMEBREW if you don't want to use gettext and msgfmt
+# installed by Homebrew.
+#
+# Define HOMEBREW_PREFIX if you have Homebrew installed in a non-default
+# location on macOS or on Linux and want to use it.
+#
 # Define NO_APPLE_COMMON_CRYPTO if you are building on Darwin/Mac OS X
 # and do not want to use Apple's CommonCrypto library.  This allows you
 # to provide your own OpenSSL library, for example from MacPorts.
@@ -1693,6 +1699,18 @@ ifeq ($(uname_S),Darwin)
 	PTHREAD_LIBS =
 endif
 
+ifndef NO_HOMEBREW
+ifdef HOMEBREW_PREFIX
+ifeq ($(shell test -d $(HOMEBREW_PREFIX)/opt/gettext && echo y),y)
+	BASIC_CFLAGS += -I$(HOMEBREW_PREFIX)/opt/gettext/include
+	BASIC_LDFLAGS += -L$(HOMEBREW_PREFIX)/opt/gettext/lib
+endif
+ifeq ($(shell test -x $(HOMEBREW_PREFIX)/opt/gettext/msgfmt && echo y),y)
+	MSGFMT = $(HOMEBREW_PREFIX)/opt/gettext/msgfmt
+endif
+endif
+endif
+
 ifdef NO_LIBGEN_H
 	COMPAT_CFLAGS += -DNO_LIBGEN_H
 	COMPAT_OBJS += compat/basename.o
diff --git a/config.mak.uname b/config.mak.uname
index 1691c6ae6e..db2a922751 100644
--- a/config.mak.uname
+++ b/config.mak.uname
@@ -149,28 +149,10 @@ ifeq ($(uname_S),Darwin)
 	CSPRNG_METHOD = arc4random
 	USE_ENHANCED_BASIC_REGULAR_EXPRESSIONS = YesPlease
 
-	# Workaround for `gettext` being keg-only and not even being linked via
-	# `brew link --force gettext`, should be obsolete as of
-	# https://github.com/Homebrew/homebrew-core/pull/53489
-        ifeq ($(shell test -d /usr/local/opt/gettext/ && echo y),y)
-		BASIC_CFLAGS += -I/usr/local/include -I/usr/local/opt/gettext/include
-		BASIC_LDFLAGS += -L/usr/local/lib -L/usr/local/opt/gettext/lib
-                ifeq ($(shell test -x /usr/local/opt/gettext/bin/msgfmt && echo y),y)
-			MSGFMT = /usr/local/opt/gettext/bin/msgfmt
-                endif
-	# On newer ARM-based machines the default installation path has changed to
-	# /opt/homebrew. Include it in our search paths so that the user does not
-	# have to configure this manually.
-	#
-	# Note that we do not employ the same workaround as above where we manually
-	# add gettext. The issue was fixed more than three years ago by now, and at
-	# that point there haven't been any ARM-based Macs yet.
-        else ifeq ($(shell test -d /opt/homebrew/ && echo y),y)
-		BASIC_CFLAGS += -I/opt/homebrew/include
-		BASIC_LDFLAGS += -L/opt/homebrew/lib
-                ifeq ($(shell test -x /opt/homebrew/bin/msgfmt && echo y),y)
-			MSGFMT = /opt/homebrew/bin/msgfmt
-                endif
+        ifeq ($(uname_M),arm64)
+		HOMEBREW_PREFIX = /opt/homebrew
+        else
+		HOMEBREW_PREFIX = /usr/local
         endif
 
 	# The builtin FSMonitor on MacOS builds upon Simple-IPC.  Both require
-- 
2.52.0

[PATCH v4 2/2] macOS: use iconv from Homebrew if needed and present

[René Scharfe]

The library function iconv(3) supplied with macOS versions 15.7.2
(Sequoia) and 26.1 (Tahoe) is unreliable when doing conversions from
ISO-2022-JP to UTF-8 in multiple steps; t3900 reports this breakage:

  not ok 17 - ISO-2022-JP should be shown in UTF-8 now
  not ok 25 - ISO-2022-JP should be shown in UTF-8 now
  not ok 38 - commit --fixup into ISO-2022-JP from UTF-8

As a workaround, use libiconv from Homebrew, if available.  Search it in
its default locations: /opt/homebrew for Apple Silicon and /usr/local
for macOS Intel, with the former taking precedence.  Respect ICONVDIR if
already set by the user, though.

Helped-by: Koji Nakamaru <koji.nakamaru@gree.net>
Signed-off-by: René Scharfe <l.s.r@web.de>
---
 Makefile         | 12 ++++++++++--
 config.mak.uname |  4 ++++
 2 files changed, 14 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 9aef22c032..b7eba509c6 100644
--- a/Makefile
+++ b/Makefile
@@ -101,12 +101,15 @@ include shared.mak
 # specify your own (or DarwinPort's) include directories and
 # library directories by defining CFLAGS and LDFLAGS appropriately.
 #
-# Define NO_HOMEBREW if you don't want to use gettext and msgfmt
-# installed by Homebrew.
+# Define NO_HOMEBREW if you don't want to use gettext, libiconv and
+# msgfmt installed by Homebrew.
 #
 # Define HOMEBREW_PREFIX if you have Homebrew installed in a non-default
 # location on macOS or on Linux and want to use it.
 #
+# Define USE_HOMEBREW_LIBICONV to link against libiconv installed by
+# Homebrew, if present.
+#
 # Define NO_APPLE_COMMON_CRYPTO if you are building on Darwin/Mac OS X
 # and do not want to use Apple's CommonCrypto library.  This allows you
 # to provide your own OpenSSL library, for example from MacPorts.
@@ -1708,6 +1711,11 @@ endif
 ifeq ($(shell test -x $(HOMEBREW_PREFIX)/opt/gettext/msgfmt && echo y),y)
 	MSGFMT = $(HOMEBREW_PREFIX)/opt/gettext/msgfmt
 endif
+ifdef USE_HOMEBREW_LIBICONV
+ifeq ($(shell test -d $(HOMEBREW_PREFIX)/opt/libiconv && echo y),y)
+	ICONVDIR ?= $(HOMEBREW_PREFIX)/opt/libiconv
+endif
+endif
 endif
 endif
 
diff --git a/config.mak.uname b/config.mak.uname
index db2a922751..38b35af366 100644
--- a/config.mak.uname
+++ b/config.mak.uname
@@ -124,6 +124,7 @@ ifeq ($(uname_S),Darwin)
 	# - MacOS 10.0.* and MacOS 10.1.0 = Darwin 1.*
 	# - MacOS 10.x.* = Darwin (x+4).* for (1 <= x)
 	# i.e. "begins with [15678] and a dot" means "10.4.* or older".
+	DARWIN_MAJOR_VERSION = $(shell expr "$(uname_R)" : '\([0-9]*\)\.')
         ifeq ($(shell expr "$(uname_R)" : '[15678]\.'),2)
 		OLD_ICONV = UnfortunatelyYes
 		NO_APPLE_COMMON_CRYPTO = YesPlease
@@ -154,6 +155,9 @@ ifeq ($(uname_S),Darwin)
         else
 		HOMEBREW_PREFIX = /usr/local
         endif
+        ifeq ($(shell test "$(DARWIN_MAJOR_VERSION)" -ge 24 && echo 1),1)
+		USE_HOMEBREW_LIBICONV = UnfortunatelyYes
+        endif
 
 	# The builtin FSMonitor on MacOS builds upon Simple-IPC.  Both require
 	# Unix domain sockets and PThreads.
-- 
2.52.0