Cooking of the ab/i18n series

Cooking of the ab/i18n series

[Ævar Arnfjörð Bjarmason]

On Wed, Aug 4, 2010 at 22:24, Junio C Hamano <gitster@pobox.com> wrote:

Since we're asking about the status of series...

> * tr/ab-i18n-fix (2010-07-25) 1 commit
>  - tests: locate i18n lib&data correctly under --valgrind
>  (this branch uses ab/i18n.)

> * ab/i18n (2010-07-19) 2 commits
>  - tests: rename test to work around GNU gettext bug
>  - Add infrastructure for translating Git with gettext
>  (this branch is used by tr/ab-i18n-fix.)

Do you have any plans for when to merge the i18n series?

It's been cooking for a while now, and it'll need a lot of follow-up
work (gettextizing) once it gets merged.

I don't have infinite time to do that, so sooner rather than later
would be better if we're going to e.g. have a fully localized Git 1.8.

I haven't been sending these follow up patches due to the reasons I
already cited. They're pretty much guaranteed to conflict with other
things in-flight.

If you're not comfortable with merging it soon for whatever reason
then perhaps I could submit something to add a no-op N_() macro to Git
in the interim. Then we could just do s/N_/_/ on the source tree once
the real series gets merged.

Re: Cooking of the ab/i18n series

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> On Wed, Aug 4, 2010 at 22:24, Junio C Hamano <gitster@pobox.com> wrote:
>
> Since we're asking about the status of series...
>
>> * tr/ab-i18n-fix (2010-07-25) 1 commit
>>  - tests: locate i18n lib&data correctly under --valgrind
>>  (this branch uses ab/i18n.)
>
>> * ab/i18n (2010-07-19) 2 commits
>>  - tests: rename test to work around GNU gettext bug
>>  - Add infrastructure for translating Git with gettext
>>  (this branch is used by tr/ab-i18n-fix.)
>
> Do you have any plans for when to merge the i18n series?

When people see the benefit of doing so.  I currently do not see much need
for it myself but I am a minority ;-).

> It's been cooking for a while now, and it'll need a lot of follow-up
> work (gettextizing) once it gets merged.

Yeah, and s/once/before/ probably.

Re: Cooking of the ab/i18n series

[Ævar Arnfjörð Bjarmason]

On Thu, Aug 5, 2010 at 22:18, Junio C Hamano <gitster@pobox.com> wrote:
> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
>> On Wed, Aug 4, 2010 at 22:24, Junio C Hamano <gitster@pobox.com> wrote:
>>
>> Since we're asking about the status of series...
>>
>>> * tr/ab-i18n-fix (2010-07-25) 1 commit
>>>  - tests: locate i18n lib&data correctly under --valgrind
>>>  (this branch uses ab/i18n.)
>>
>>> * ab/i18n (2010-07-19) 2 commits
>>>  - tests: rename test to work around GNU gettext bug
>>>  - Add infrastructure for translating Git with gettext
>>>  (this branch is used by tr/ab-i18n-fix.)
>>
>> Do you have any plans for when to merge the i18n series?
>
> When people see the benefit of doing so.  I currently do not see much need
> for it myself but I am a minority ;-).

That's news to me. I'd assumed that it was mostly on track, i.e. that
it would get merged down after cooking for a while in pu.

However, if it's a matter of gathering popular support maybe I should
change my strategy a bit.

Perhaps it would help if I sent the entire series as it is in pu now
to the list along with a cover letter explaining the main implications
of merging it?

All the RFCs so far have either focused on discussing the idea without
the implementation, or small parts of the implementation that needed
to be improved.

>> It's been cooking for a while now, and it'll need a lot of follow-up
>> work (gettextizing) once it gets merged.
>
> Yeah, and s/once/before/ probably.

What sort of work do you think is needed before it's merged? It seems
to Just Work everywhere I and others have tested it (sans some minor
patches like working around old gettext bugs).

The follow-up work I was referring to was the project we'll need to
undertake once it's merged to convert "foo" to _("foo") as
appropriate.

What sort of work do you think might be needed before it's merged?

Re: Cooking of the ab/i18n series

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> On Thu, Aug 5, 2010 at 22:18, Junio C Hamano <gitster@pobox.com> wrote:
>>
>> When people see the benefit of doing so.  I currently do not see much need
>> for it myself but I am a minority ;-).
>
> That's news to me. I'd assumed that it was mostly on track, i.e. that
> it would get merged down after cooking for a while in pu.
>
> However, if it's a matter of gathering popular support maybe I should
> change my strategy a bit.

The "popular support" needs to be qualified.  If you ask any random person
"Is it a good thing if software package X supports i18n?", the answer
would always be "yes"; popular support in that sense doesn't mean much.

I am more worried about unintended consequence of this particular
execution.  For example, I would want to be absolutely sure that we won't
break plumbing output in 'next' and the proposed mechanism helps others
avoid breaking things by mistake.

> The follow-up work I was referring to was the project we'll need to
> undertake once it's merged to convert "foo" to _("foo") as
> appropriate.

That is one good example.  Perhaps we can get a list of messages that we
can place in Documentation/ area (e.g. "'Not up-to-date' - this message is
given when you have local changes in a file in the working tree; given by
command X, Y and Z") out of that effort for free?  Perhaps such a list can
help us verify that i18n does not break plumbing output (because the list
does not contain plumbing messages)?

Re: Cooking of the ab/i18n series

[Ævar Arnfjörð Bjarmason]

On Fri, Aug 6, 2010 at 13:30, Junio C Hamano <gitster@pobox.com> wrote:
> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
>> On Thu, Aug 5, 2010 at 22:18, Junio C Hamano <gitster@pobox.com> wrote:
>>>
>>> When people see the benefit of doing so.  I currently do not see much need
>>> for it myself but I am a minority ;-).
>>
>> That's news to me. I'd assumed that it was mostly on track, i.e. that
>> it would get merged down after cooking for a while in pu.
>>
>> However, if it's a matter of gathering popular support maybe I should
>> change my strategy a bit.
>
> The "popular support" needs to be qualified.  If you ask any random person
> "Is it a good thing if software package X supports i18n?", the answer
> would always be "yes"; popular support in that sense doesn't mean much.

I was thinking about support from the core contributors, which'd have
to deal with gettext in the long term in one way or another.

> I am more worried about unintended consequence of this particular
> execution.  For example, I would want to be absolutely sure that we won't
> break plumbing output in 'next' and the proposed mechanism helps others
> avoid breaking things by mistake.

I'm also worried about that, and I have some plans to deal with it
after the merge.

The first and most obvious one is that the list will be reviewing
gettexizing patches as they go through. A patch which changes some
plumbing format would be called out, but not one that just changes the
UI messsage of e.g. "git init".

There's also more that can be done, e.g. altering the test-lib.sh so
that you can set an environment variable that causes it not to reset
LC_ALL to C. Then run the tests and see if anything breaks.

I was going to run a smoker with that setup on some of the major
languages if gettext and smoke support was accepted.

>> The follow-up work I was referring to was the project we'll need to
>> undertake once it's merged to convert "foo" to _("foo") as
>> appropriate.
>
> That is one good example.  Perhaps we can get a list of messages that we
> can place in Documentation/ area (e.g. "'Not up-to-date' - this message is
> given when you have local changes in a file in the working tree; given by
> command X, Y and Z") out of that effort for free?  Perhaps such a list can
> help us verify that i18n does not break plumbing output (because the list
> does not contain plumbing messages)?

We'd get this sort of list out of "TRANSLATORS:" comments for
free. They're automatically extracted and presented to translators and
others with the xgettext program.

Maintaining a list of messages in Documentation/ somewhere that's
bound to get out of date with the source code doesn't make sense given
the TRANSLATORS support.

Re: Cooking of the ab/i18n series

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> On Fri, Aug 6, 2010 at 13:30, Junio C Hamano <gitster@pobox.com> wrote:
>> That is one good example.  Perhaps we can get a list of messages that we
>> can place in Documentation/ area (e.g. "'Not up-to-date' - this message is
>> given when you have local changes in a file in the working tree; given by
>> command X, Y and Z") out of that effort for free?  Perhaps such a list can
>> help us verify that i18n does not break plumbing output (because the list
>> does not contain plumbing messages)?
>
> We'd get this sort of list out of "TRANSLATORS:" comments for
> free. They're automatically extracted and presented to translators and
> others with the xgettext program.
>
> Maintaining a list of messages in Documentation/ somewhere that's
> bound to get out of date with the source code doesn't make sense given
> the TRANSLATORS support.

Read it again---I didn't say "maintaining" a list at all; we are saying
the same thing more or less ;-)  You might need to massage the output from
TRANSLATORS: comment to a more readable form suitable for inclusion in the
documentation (depending on how that thing looks like), though.

Re: Cooking of the ab/i18n series

[Ævar Arnfjörð Bjarmason]

On Fri, Aug 6, 2010 at 15:48, Junio C Hamano <gitster@pobox.com> wrote:
> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
>> On Fri, Aug 6, 2010 at 13:30, Junio C Hamano <gitster@pobox.com> wrote:
>>> That is one good example.  Perhaps we can get a list of messages that we
>>> can place in Documentation/ area (e.g. "'Not up-to-date' - this message is
>>> given when you have local changes in a file in the working tree; given by
>>> command X, Y and Z") out of that effort for free?  Perhaps such a list can
>>> help us verify that i18n does not break plumbing output (because the list
>>> does not contain plumbing messages)?
>>
>> We'd get this sort of list out of "TRANSLATORS:" comments for
>> free. They're automatically extracted and presented to translators and
>> others with the xgettext program.
>>
>> Maintaining a list of messages in Documentation/ somewhere that's
>> bound to get out of date with the source code doesn't make sense given
>> the TRANSLATORS support.
>
> Read it again---I didn't say "maintaining" a list at all; we are saying
> the same thing more or less ;-)  You might need to massage the output from
> TRANSLATORS: comment to a more readable form suitable for inclusion in the
> documentation (depending on how that thing looks like), though.

Ah yes. I didn't read it carefully enough. Yeah, if we're just talking
about autogeneration it isn't hard to get a list of all translatable
strings into a single ASCIIDOC page.

I plan to write some translator documentation once this gets in and we
start having people submit translations. E.g. a document that lists
some of the core Git concepts that the translator will need to
translate (e.g. "pull", "push", "commit", "tree", "object" etc.).

Re: Cooking of the ab/i18n series

[Ævar Arnfjörð Bjarmason]

On Fri, Aug 6, 2010 at 17:01, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> On Fri, Aug 6, 2010 at 15:48, Junio C Hamano <gitster@pobox.com> wrote:
> [...]

I don't mean to bombard you with E-Mails, but hopefully this one is a
bit more to the point.

As I'm sure you've gathered by now I'm keen to get this into Git. I'm
also fully prepared to address any specific concerns about the series
before it gets merged.

However, and maybe I'm just dense, I can't really see some unambiguous
things about the series that I could improve from your comments. So
let's try to clear them up, shall we?

My mental plan for this series has basically been as follows:

  1. Get it to a state where it can cook in pu [You Are Here]

  2. After it's been there for a while get it to master

  3. Once it's there for a while and we're sure the new dependency /
     code doesn't harm some more obscure systems..

  4. Start submitting patches to the main porcelain $(grep
     'mainporcelain common' command-list.txt) to make the most common
     user-visible messages translatable.

  5. Recruit translators to translate the strings in #4. Send
     translations in as patches adding/altering the *.po files.

While doing #4 and #5 I planned to write some new
documentation. E.g. a quick guide for Git hackers showing how you can
add strings that are properly i18n-able. And for #5 an equivalent
thing for non-techie translators showing what the need to translate,
how to submit translations etc.

Now, your main concern is that this doesn't break plumbing output. My
plan for that was to just leave it for later. As long as I'm not
altering something in 'mainporcelain common' I'm not going to break
the plumbing.

I think I can read between the lines that you're uneasy about this
approach. But I'm not sure how else to handle it.

One thing that could perhaps help things in the long run would be to
explicitly mark plumbing messages as not for translation. E.g. with a
"commit" -> P_("commit") macro.

That'd also give us something like the Documentation/ page you
suggest, but with only the plumbing messages extracted on a per-tool
basis (using the gettext tools). That'd probably make for a useful API
reference.

To sum it up. I'm happy to amend something in the 5-point plan above,
or to write new code to make the gettext series more acceptable. I
just don't see what that something is.

In any case, thanks for all the work you and others have put into
getting the series into the state it's in today. I really appreciate it.

Re: Cooking of the ab/i18n series

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> My mental plan for this series has basically been as follows:
>
>   1. Get it to a state where it can cook in pu [You Are Here]
>
>   2. After it's been there for a while get it to master
>
>   3. Once it's there for a while and we're sure the new dependency /
>      code doesn't harm some more obscure systems..
>
>   4. Start submitting patches to the main porcelain $(grep
>      'mainporcelain common' command-list.txt) to make the most common
>      user-visible messages translatable.
>
>   5. Recruit translators to translate the strings in #4. Send
>      translations in as patches adding/altering the *.po files.

Matches my expectations modulo s/master/next/.  The stuff parked in 'pu'
was primarily because most of the work to get to this point was done
during the pre-release freeze for 1.7.2 and I didn't want to get
distracted, while I obviously did not lose patches.

> Now, your main concern is that this doesn't break plumbing output....

That's not 'main'.

'Breaking plumbing' is merely an example of a larger 'main concern' which
is 'unintended consequences'.

And please don't ask me to enumerate them exhaustively.  By definition
'unintended consequences' cannot be enumerated.  They are discovered over
time either by code inspection, by people guinea-pigging a version outside
'master', and by careful thinking.

Re: Cooking of the ab/i18n series

[Ævar Arnfjörð Bjarmason]

On Fri, Aug 6, 2010 at 23:01, Junio C Hamano <gitster@pobox.com> wrote:
> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
>> My mental plan for this series has basically been as follows:
>>
>>   1. Get it to a state where it can cook in pu [You Are Here]
>>
>>   2. After it's been there for a while get it to master
>>
>>   3. Once it's there for a while and we're sure the new dependency /
>>      code doesn't harm some more obscure systems..
>>
>>   4. Start submitting patches to the main porcelain $(grep
>>      'mainporcelain common' command-list.txt) to make the most common
>>      user-visible messages translatable.
>>
>>   5. Recruit translators to translate the strings in #4. Send
>>      translations in as patches adding/altering the *.po files.
>
> Matches my expectations modulo s/master/next/.

Yes, I meant to say "on track to master" (via next).

> The stuff parked in 'pu' was primarily because most of the work to
> get to this point was done during the pre-release freeze for 1.7.2
> and I didn't want to get distracted, while I obviously did not lose
> patches.

I didn't mean to rush you, the delay is fine. It just sounded like the
series might be stuck in pu in perpetuity from the initial "When
people see the benefit of doing so" comment.

But if not, nevermind.

>> Now, your main concern is that this doesn't break plumbing output....
>
> That's not 'main'.
>
> 'Breaking plumbing' is merely an example of a larger 'main concern' which
> is 'unintended consequences'.

Sure, all code's going to have problems, and the ab/i18n series is
around 1300 lines, so I'd be quite surprised if it didn't have some
bug.

> And please don't ask me to enumerate them exhaustively.  By definition
> 'unintended consequences' cannot be enumerated.  They are discovered over
> time either by code inspection, by people guinea-pigging a version outside
> 'master', and by careful thinking.

I was just wondering if there was anything particular that you were
worried about that could be addressed.

But as you've clarified there's nothing about it that I should be
explicitly addressing before it can move forward (if I've understood
you correctly).

Thanks for clarifying that and for your time.