Re: Re*: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

["Torsten Bögershausen" <tboegi@web.de>]

On 03/09/2012 11:30 PM, Junio C Hamano wrote:
> Junio C Hamano<gitster@pobox.com>  writes:
>
>> I would be OK if the patch read like the attached, but I do not
>> think treating the "show key" and "stuff with ?" case differently
>> like the original code did makes any sense.
> Ok, here is a re-roll with proper-ish log message.  I've tested this
> only in one of my VMs with is_IS locale (which happened to be Fedora
> 15).
>
> -- >8 --
> Subject: [PATCH] t0204: clarify the "observe undefined behaviour" test
>
> This test asks for an impossible conversion to the system by
> preparing an UTF-8 translation with characters that cannot be
> expressed in ISO-8859-1, and then asking the message shown in
> ISO-8859-1.  Even though the behaviour against such a request is
> undefined, it may be interesting to see what the system does, and
> the purpose of this test is to see if there are platforms that
> exhibit behaviour that we haven't seen.
>
> The original recognized two known modes of behaviour:
>
>   - the key used to query the message catalog ("TEST: Old English
>     Runes"), saying "I cannot do that i18n".
>   - impossible characters replaced with ASCII "?", saying "I punt".
>
> but they were treated totally differently.  The test simply issued
> an informational message "Your system punts on this one" for the
> first error mode, while it diagnosed the latter as "Your system is
> good; you pass!".
>
> It turns out that Mac OS X exhibits a third mode of error behaviour,
> to spew out the raw value stored in the message catalog.  The test
> diagnosed this behaviour as "broken", but it is merely trying to do
> its best to respond to an impossible request by saying "I punt" in a
> way that is slightly different from the second one.
>
> Update the offending test to make it clear what is (and is not)
> being tested, update the code structure so that newly discovered
> error mode can easily be added to it later, and reword the message
> that comes from a failing case to clarify that it is not the system
> that is broken when it fails, but merely that the behaviour is not
> something we have seen.
>
> Signed-off-by: Junio C Hamano<gitster@pobox.com>
> ---
>   t/t0204-gettext-reencode-sanity.sh |   37 ++++++++++++++++++++++--------------
>   1 file changed, 23 insertions(+), 14 deletions(-)
>
> diff --git a/t/t0204-gettext-reencode-sanity.sh b/t/t0204-gettext-reencode-sanity.sh
> index 189af90..8437e51 100755
> --- a/t/t0204-gettext-reencode-sanity.sh
> +++ b/t/t0204-gettext-reencode-sanity.sh
> @@ -7,6 +7,10 @@ test_description="Gettext reencoding of our *.po/*.mo files works"
>
>   . ./lib-gettext.sh
>
> +# The constants used in a tricky observation for undefined behaviour
> +RUNES="TILRAUN: ᚻᛖ ᚳᚹᚫᚦ ᚦᚫᛏ ᚻᛖ ᛒᚢᛞᛖ ᚩᚾ ᚦᚫᛗ ᛚᚪᚾᛞᛖ ᚾᚩᚱᚦᚹᛖᚪᚱᛞᚢᛗ ᚹᛁᚦ ᚦᚪ ᚹᛖᛥᚫ"
> +PUNTS="TILRAUN: ?? ???? ??? ?? ???? ?? ??? ????? ??????????? ??? ?? ????"
> +MSGKEY="TEST: Old English Runes"
>
>   test_expect_success GETTEXT_LOCALE 'gettext: Emitting UTF-8 from our UTF-8 *.mo files / Icelandic' '
>       printf "TILRAUN: Halló Heimur!">expect&&
> @@ -15,8 +19,8 @@ test_expect_success GETTEXT_LOCALE 'gettext: Emitting UTF-8 from our UTF-8 *.mo
>   '
>
>   test_expect_success GETTEXT_LOCALE 'gettext: Emitting UTF-8 from our UTF-8 *.mo files / Runes' '
> -    printf "TILRAUN: ᚻᛖ ᚳᚹᚫᚦ ᚦᚫᛏ ᚻᛖ ᛒᚢᛞᛖ ᚩᚾ ᚦᚫᛗ ᛚᚪᚾᛞᛖ ᚾᚩᚱᚦᚹᛖᚪᚱᛞᚢᛗ ᚹᛁᚦ ᚦᚪ ᚹᛖᛥᚫ">expect&&
> -    LANGUAGE=is LC_ALL="$is_IS_locale" gettext "TEST: Old English Runes">actual&&
> +    printf "%s" "$RUNES">expect&&
> +    LANGUAGE=is LC_ALL="$is_IS_locale" gettext "$MSGKEY">actual&&
>       test_cmp expect actual
>   '
>
> @@ -26,18 +30,23 @@ test_expect_success GETTEXT_ISO_LOCALE 'gettext: Emitting ISO-8859-1 from our UT
>       test_cmp expect actual
>   '
>
> -test_expect_success GETTEXT_ISO_LOCALE 'gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Runes' '
> -    LANGUAGE=is LC_ALL="$is_IS_iso_locale" gettext "TEST: Old English Runes">runes&&
> -
> -	if grep "^TEST: Old English Runes$" runes
> -	then
> -		say "Your system can not handle this complexity and returns the string as-is"
> -	else
> -		# Both Solaris and GNU libintl will return this stream of
> -		# question marks, so it is s probably portable enough
> -		printf "TILRAUN: ?? ???? ??? ?? ???? ?? ??? ????? ??????????? ??? ?? ????">runes-expect&&
> -		test_cmp runes-expect runes
> -	fi
> +test_expect_success GETTEXT_ISO_LOCALE 'gettext: impossible ISO-8859-1 output' '
> +	LANGUAGE=is LC_ALL="$is_IS_iso_locale" gettext "$MSGKEY">runes&&
> +	case "$(cat runes)" in
> +	"$MSGKEY")
> +		say "Your system gives back the key to message catalog"
> +		;;
> +	"$PUNTS")
> +		say "Your system replaces an impossible character with ?"
> +		;;
> +	"$RUNES")
> +		say "Your system gives back the raw message for an impossible request"
> +		;;
> +	*)
> +		say "We never saw the error behaviour your system exhibits"
> +		false
> +		;;
> +	esac
>   '
>
>   test_expect_success GETTEXT_LOCALE 'gettext: Fetching a UTF-8 msgid ->  UTF-8' '
Hi Junio and all,
Thanks, the suggestion looks good to me.

I want be able to test it, but there is one problem:
I can not reproduce the original problem any more.
I went through all commit, all my branches, run git bisect, restore old 
backups:
The problem is gone.

Then I diged into the test code. As Hannes pointed out, all test cases 
in t0204 are
skipped.
This is not because Mac OS X does not have the island locale (it has),
but because GIT_INTERNAL_GETTEXT_SH_SCHEME" = fallthrough.

Side note: I will send an improved lib-gettext.sh as a suggested patch.

Back to my setup:
I realized, that there is i18n stuff installed by git under $HOME/share.
Could that be a reason why I got that non-reproducable failure in t0204?

And the skipped test cases seem to indicate that the gettext feature is
not working under Mac OS X?

Does anybody else have t0204 working and not skipped?
Any hints are welcome.

/Torsten