localization of Grub

localization of Grub

[Carles Pina i Estany]

Hello,

I just was wondering: what are your thoughts about localization and
Grub?

We don't have nothing because it's a decision (I would undertand) or we
have not done it?

Thanks,

-- 
Carles Pina i Estany		GPG id: 0x17756391
	http://pinux.info

Re: localization of Grub

[Carles Pina i Estany]

Hello,

On Sep/26/2008, Carles Pina i Estany wrote:

> I just was wondering: what are your thoughts about localization and
> Grub?

Just to clarify: translate user strings, mainly in the booting screen
(or make mainly all streams part of a theme, etc. etc.)

> We don't have nothing because it's a decision (I would undertand) or we

undertand -> understand :-)

-- 
Carles Pina i Estany		GPG id: 0x17756391
	http://pinux.info

Re: localization of Grub

[Felix Zielcke]

Am Freitag, den 26.09.2008, 10:34 +0200 schrieb Carles Pina i Estany:
> Hello,
> 
> On Sep/26/2008, Carles Pina i Estany wrote:
> 
> > I just was wondering: what are your thoughts about localization and
> > Grub?
> 
> Just to clarify: translate user strings, mainly in the booting screen
> (or make mainly all streams part of a theme, etc. etc.)
> 
> > We don't have nothing because it's a decision (I would undertand) or we

For this wish we have a Debian bugreport, which is even over 9 years
old.

http://bugs.debian.org/33723

This is wanted but nobody did work on this yet afaik.
It would be good if there would be multiple keyboard layouts too I
think.

Re: localization of Grub

[Carles Pina i Estany]

Hi,

On Sep/26/2008, Felix Zielcke wrote:
> Am Freitag, den 26.09.2008, 10:34 +0200 schrieb Carles Pina i Estany:
> > Hello,
> > 
> > On Sep/26/2008, Carles Pina i Estany wrote:
> > 
> > > I just was wondering: what are your thoughts about localization and
> > > Grub?
> > 
> > Just to clarify: translate user strings, mainly in the booting screen
> > (or make mainly all streams part of a theme, etc. etc.)
> > 
> > > We don't have nothing because it's a decision (I would undertand) or we
> 
> For this wish we have a Debian bugreport, which is even over 9 years
> old.
> 
> http://bugs.debian.org/33723
> 
> This is wanted but nobody did work on this yet afaik.
> It would be good if there would be multiple keyboard layouts too I
> think.

I think that different keyboard layouts it's even more important than
different languages...

Do we have any plan? Maybe I will take a look (but during 20 days in
october I will not have lot of time :-( so not sure if I will do it
after October or not, I will see...)

Any suggestion for the implementation? How user would change the
keyboard layout? Or maybe at beginning some setting (variable) at
grub.cfg?

-- 
Carles Pina i Estany		GPG id: 0x17756391
	http://pinux.info

Re: localization of Grub

[Robert Millan]

On Fri, Sep 26, 2008 at 02:19:09PM +0200, Carles Pina i Estany wrote:
> > 
> > This is wanted but nobody did work on this yet afaik.
> > It would be good if there would be multiple keyboard layouts too I
> > think.
> 
> I think that different keyboard layouts it's even more important than
> different languages...
> 
> Do we have any plan? Maybe I will take a look (but during 20 days in
> october I will not have lot of time :-( so not sure if I will do it
> after October or not, I will see...)
> 
> Any suggestion for the implementation? How user would change the
> keyboard layout? Or maybe at beginning some setting (variable) at
> grub.cfg?

Hi,

It's very nice you want to work on localisation.  This is a very welcome
feature (and I think it's been discussed before).

(I assume we're talking about a gettext equivalent rather than keyboard
layouts, which is a separate -but also nice- feature)

My recommendation for a roadmap (please comment/discuss/challenge/etc!) would
be:

  - gettextise the util tools, so they can be translated as normal programs.

  - add support to grub (a gettext module or so) so that it can load our .mo
    files and obtain strings from it, by implementing gettext() (aka _()).

I'd also recommend making the user interface similar to posix locales, for
consistency's sake (which you know I hold dearly ;-)).

Even if you only have time/interest to implement the first part, this is
already a significant benefit.  Keep in mind that grub.cfg generation is done
by update-grub which happens entirely on userland, so we get translated menu
entries that way.

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Re: localization of Grub

[Carles Pina i Estany]

Hello,

On Sep/28/2008, Robert Millan wrote:
> On Fri, Sep 26, 2008 at 02:19:09PM +0200, Carles Pina i Estany wrote:
> > > 
> > > This is wanted but nobody did work on this yet afaik.
> > > It would be good if there would be multiple keyboard layouts too I
> > > think.
> > 
> > I think that different keyboard layouts it's even more important than
> > different languages...
> > 
> > Do we have any plan? Maybe I will take a look (but during 20 days in
> > october I will not have lot of time :-( so not sure if I will do it
> > after October or not, I will see...)
> > 
> > Any suggestion for the implementation? How user would change the
> > keyboard layout? Or maybe at beginning some setting (variable) at
> > grub.cfg?
> 
> Hi,
> 
> It's very nice you want to work on localisation.  This is a very welcome
> feature (and I think it's been discussed before).
> 
> (I assume we're talking about a gettext equivalent rather than keyboard
> layouts, which is a separate -but also nice- feature)

Yes, I was talking about gettext/equivalent (I will send another email
talking about keyboard layouts)

> My recommendation for a roadmap (please comment/discuss/challenge/etc!) would
> be:
> 
>   - gettextise the util tools, so they can be translated as normal
>   programs.

ok! (I guess/hope that will be more "burocratic" work than new work)

>   - add support to grub (a gettext module or so) so that it can load our .mo
>     files and obtain strings from it, by implementing gettext() (aka _()).

this is the interesting part, I think :-)

> I'd also recommend making the user interface similar to posix locales, for
> consistency's sake (which you know I hold dearly ;-)).

I know!

> Even if you only have time/interest to implement the first part, this
> is already a significant benefit.  Keep in mind that grub.cfg
> generation is done by update-grub which happens entirely on userland,
> so we get translated menu entries that way.

I don't expect to have time until 22th October (aprox.). I have more
interest in the second part (it's "newer") than first part, but first
part has a practical and fast effects with (I think) less investment.

I think that I will do both things, but will take some time. If somebody
has time/interest to do it before: please, raise your hand!

-- 
Carles Pina i Estany		GPG id: 0x17756391
	http://pinux.info

Re: localization of Grub

[Robert Millan]

On Sun, Sep 28, 2008 at 11:49:54PM +0200, Carles Pina i Estany wrote:
> >   - gettextise the util tools, so they can be translated as normal
> >   programs.
> 
> ok! (I guess/hope that will be more "burocratic" work than new work)

It _is_ technical work I think, but less fun than the other part :-/

> I don't expect to have time until 22th October (aprox.). I have more
> interest in the second part (it's "newer") than first part, but first
> part has a practical and fast effects with (I think) less investment.

The second part also builds on the first, to some extent.  I.e. if you want
to test gettext support in GRUB itself, you need some strings to translate,
and these are provided by the .mo files which are only built if the build
system supports that, etc.

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Re: localization of Grub

[Vesa Jääskeläinen]

Robert Millan wrote:
> On Sun, Sep 28, 2008 at 11:49:54PM +0200, Carles Pina i Estany wrote:
>>>   - gettextise the util tools, so they can be translated as normal
>>>   programs.
>> ok! (I guess/hope that will be more "burocratic" work than new work)
> 
> It _is_ technical work I think, but less fun than the other part :-/
> 
>> I don't expect to have time until 22th October (aprox.). I have more
>> interest in the second part (it's "newer") than first part, but first
>> part has a practical and fast effects with (I think) less investment.
> 
> The second part also builds on the first, to some extent.  I.e. if you want
> to test gettext support in GRUB itself, you need some strings to translate,
> and these are provided by the .mo files which are only built if the build
> system supports that, etc.

Before you guy's go too deep in detail I would like to remind special
requirements for graphical user interface related to localization.

You can't just printf stuff to screen in there. There has to be some
changes in logic how information is presented to user. Currently there
is lot of printf's in the code to display information and that is not
going to be too pretty for graphical menu as we need to display some
kind of console on event when there is something to be displayed.

Also try to think how different languages differentiate for displaying
certain types of information. Here is some simple example. (Bear in mind
if there are grammatical errors or typos or so :))

(eng) "See Figure 2 in page 14 for more details.' -> (fin) "Sivulla 14
olevassa kuvassa 2 on enemmän tietoa asiasta.'

Please notice difference in order of arguments in the languages.

Also there are some weird scripts that change order of characters. In
example for some right-to-left scripts seem to have this feature. On
example that I think belongs to this group is hebrew where they normally
write from right-to-left, but for English (or foreign) texts are still
visually correct.

In example:
"<native>This is native 1</native><english>2and this is
English3</english><native>4, so weird.</native>"

Would be something like:
".rdiew os ,42and this is English31 evitan si sihT"

So this subject really has more details than meets the eye in first sight :)

Re: localization of Grub

[Javier Martín]

[-- Attachment #1: Type: text/plain, Size: 2846 bytes --]

El lun, 29-09-2008 a las 21:48 +0300, Vesa Jääskeläinen escribió:
> Robert Millan wrote:
> > On Sun, Sep 28, 2008 at 11:49:54PM +0200, Carles Pina i Estany wrote:
> >>>   - gettextise the util tools, so they can be translated as normal
> >>>   programs.
> >> ok! (I guess/hope that will be more "burocratic" work than new work)
> > 
> > It _is_ technical work I think, but less fun than the other part :-/
> > 
> >> I don't expect to have time until 22th October (aprox.). I have more
> >> interest in the second part (it's "newer") than first part, but first
> >> part has a practical and fast effects with (I think) less investment.
> > 
> > The second part also builds on the first, to some extent.  I.e. if you want
> > to test gettext support in GRUB itself, you need some strings to translate,
> > and these are provided by the .mo files which are only built if the build
> > system supports that, etc.
> 
> Before you guy's go too deep in detail I would like to remind special
> requirements for graphical user interface related to localization.
> 
> You can't just printf stuff to screen in there. There has to be some
> changes in logic how information is presented to user. Currently there
> is lot of printf's in the code to display information and that is not
> going to be too pretty for graphical menu as we need to display some
> kind of console on event when there is something to be displayed.
> 
> Also try to think how different languages differentiate for displaying
> certain types of information. Here is some simple example. (Bear in mind
> if there are grammatical errors or typos or so :))
> 
> (eng) "See Figure 2 in page 14 for more details.' -> (fin) "Sivulla 14
> olevassa kuvassa 2 on enemmän tietoa asiasta.'
> 
> Please notice difference in order of arguments in the languages.
> 
> Also there are some weird scripts that change order of characters. In
> example for some right-to-left scripts seem to have this feature. On
> example that I think belongs to this group is hebrew where they normally
> write from right-to-left, but for English (or foreign) texts are still
> visually correct.
> 
> In example:
> "<native>This is native 1</native><english>2and this is
> English3</english><native>4, so weird.</native>"
> 
> Would be something like:
> ".rdiew os ,42and this is English31 evitan si sihT"
> 
> So this subject really has more details than meets the eye in first sight :)
> 
Well, no one proposed a full implementation of the Unicode bi-di
algorithm, and I think such a complex feat goes way beyond the
development power behind GRUB right now. However, gettext does allow for
the changes in the argument order you are worried about. Thus, there
should be no problem translating the interface to LTR languages for the
time being.

-Habbit

[-- Attachment #2: Esta parte del mensaje está firmada digitalmente --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: localization of Grub

[Carles Pina i Estany]

Hi,

On Sep/29/2008, Robert Millan wrote:
> On Sun, Sep 28, 2008 at 11:49:54PM +0200, Carles Pina i Estany wrote:
> > >   - gettextise the util tools, so they can be translated as normal
> > >   programs.
> > 
> > ok! (I guess/hope that will be more "burocratic" work than new work)
> 
> It _is_ technical work I think, but less fun than the other part :-/

When I said burocratic it was not like "Spanish burocracy" :-)
I added locale support in another program (Python based). By
"burocratic" I was thinking the part adding _(    ) in all strings :-)

(adding locale support to a non locale program has been quite much
difficult and work... also I guess that there will be some problems
adding locale dependencies in Grub too)

> > I don't expect to have time until 22th October (aprox.). I have more
> > interest in the second part (it's "newer") than first part, but first
> > part has a practical and fast effects with (I think) less investment.
> 
> The second part also builds on the first, to some extent.  I.e. if you want
> to test gettext support in GRUB itself, you need some strings to translate,
> and these are provided by the .mo files which are only built if the build
> system supports that, etc.

I know

-- 
Carles Pina i Estany		GPG id: 0x17756391
	http://pinux.info

Re: localization of Grub

[Carles Pina i Estany]

Hello,

On Sep/29/2008, Vesa Jääskeläinen wrote:
> Robert Millan wrote:
> > On Sun, Sep 28, 2008 at 11:49:54PM +0200, Carles Pina i Estany wrote:
> >>>   - gettextise the util tools, so they can be translated as normal
> >>>   programs.
> >> ok! (I guess/hope that will be more "burocratic" work than new work)
> > 
> > It _is_ technical work I think, but less fun than the other part :-/
> > 
> >> I don't expect to have time until 22th October (aprox.). I have more
> >> interest in the second part (it's "newer") than first part, but first
> >> part has a practical and fast effects with (I think) less investment.
> > 
> > The second part also builds on the first, to some extent.  I.e. if you want
> > to test gettext support in GRUB itself, you need some strings to translate,
> > and these are provided by the .mo files which are only built if the build
> > system supports that, etc.
> 
> Before you guy's go too deep in detail I would like to remind special
> requirements for graphical user interface related to localization.

first step (as long as I understood from Robert) would be adding
localization (say... gettext support) to everything under grub2/util .

This is userspace and I hope that standard gettext will be enough for
there. What do you think? (I hope that will be enough because gettext is
used in lot of places, even when I cannot find any RTL support after a
fast Googling :-( )

> You can't just printf stuff to screen in there. There has to be some
> changes in logic how information is presented to user. Currently there
> is lot of printf's in the code to display information and that is not
> going to be too pretty for graphical menu as we need to display some
> kind of console on event when there is something to be displayed.

well, I think that this printf's will not look pretty in any language (I
mean, it's not a localization problem)

> Also try to think how different languages differentiate for displaying
> certain types of information. Here is some simple example. (Bear in mind
> if there are grammatical errors or typos or so :))
> 
> (eng) "See Figure 2 in page 14 for more details.' -> (fin) "Sivulla 14
> olevassa kuvassa 2 on enemmän tietoa asiasta.'
> 
> Please notice difference in order of arguments in the languages.

gettext supports it

> Also there are some weird scripts that change order of characters. In
> example for some right-to-left scripts seem to have this feature. On
> example that I think belongs to this group is hebrew where they normally
> write from right-to-left, but for English (or foreign) texts are still
> visually correct.
> 
> In example:
> "<native>This is native 1</native><english>2and this is
> English3</english><native>4, so weird.</native>"
> 
> Would be something like:
> ".rdiew os ,42and this is English31 evitan si sihT"
> 
> So this subject really has more details than meets the eye in first
> sight :)

RTL problem is more about presentation, no? I will check how other
programs do it. If the hebrew translator writes the correct translated
string in the .po we will have it in Grub... but the alignment would be
incorrect. It would look, for Left to Right people, like having
everything aligned to the right.

-- 
Carles Pina i Estany		GPG id: 0x17756391
	http://pinux.info

Re: localization of Grub

[Carles Pina i Estany]

Hello again,

Let me to re-open this thread, where we were discussing about
localization and translation:

On Sep/28/2008, Robert Millan wrote:
> On Fri, Sep 26, 2008 at 02:19:09PM +0200, Carles Pina i Estany wrote:
> > > 
> > > This is wanted but nobody did work on this yet afaik.
> > > It would be good if there would be multiple keyboard layouts too I
> > > think.
> > 
> > I think that different keyboard layouts it's even more important than
> > different languages...
> > 
> > Do we have any plan? Maybe I will take a look (but during 20 days in
> > october I will not have lot of time :-( so not sure if I will do it
> > after October or not, I will see...)
> > 
> > Any suggestion for the implementation? How user would change the
> > keyboard layout? Or maybe at beginning some setting (variable) at
> > grub.cfg?
> 
> Hi,
> 
> It's very nice you want to work on localisation.  This is a very welcome
> feature (and I think it's been discussed before).
> 
> (I assume we're talking about a gettext equivalent rather than keyboard
> layouts, which is a separate -but also nice- feature)
> 
> My recommendation for a roadmap (please comment/discuss/challenge/etc!) would
> be:
> 
>   - gettextise the util tools, so they can be translated as normal programs.

Do you mean the strings that are generated in standard Linux user-space
and written to grub.cfg?

What do you (everybody in the list) think about the text help that
appears here and there (output of Grub -usually shell- commands?)
Needed? Or these texts are good enough in English (like commands in the
programming languages)

>   - add support to grub (a gettext module or so) so that it can load our .mo
>     files and obtain strings from it, by implementing gettext() (aka _()).

This part is more clear (at least theoretically, of course :-) )
I understand that all error messages, messages, etc. that Grub prints
while Grub is executing should be translated.

My understanding is that we should have some module to handle some
function (eg. _()), that would read some variable to choose to which
language, if available, translate some string. Like gettext, as you
said.

> I'd also recommend making the user interface similar to posix locales, for
> consistency's sake (which you know I hold dearly ;-)).

I agree

I'm not familiar in the .mo internals, I will take a look. I understand
that this gives us quite much flexibility, if we would implement
everything (like argument orders, etc.)

> Even if you only have time/interest to implement the first part, this
> is already a significant benefit.  Keep in mind that grub.cfg
> generation is done by update-grub which happens entirely on userland,
> so we get translated menu entries that way.

As I said before by mail, grub.cfg don't have lot of UI strings.
Some, but not a lot.

I understand that we/I should adapt some scripts using, for example, this
guideline?:
http://www.gnu.org/software/automake/manual/gettext/Preparing-Shell-Scripts.html

(I haven't tested now, I think that some years ago I did some tests with
Bash and gettext)

-- 
Carles Pina i Estany		GPG id: 0x17756391
	http://pinux.info

Re: localization of Grub

[Carles Pina i Estany]

Hello,

On Dec/08/2008, Carles Pina i Estany wrote:

> I understand that we/I should adapt some scripts using, for example, this
> guideline?:
> http://www.gnu.org/software/automake/manual/gettext/Preparing-Shell-Scripts.html
> 
> (I haven't tested now, I think that some years ago I did some tests with

They use gettext.sh file, in my Debian is in:
/usr/bin/gettext.sh, in package gettext-base . Do you think that this is
the best way? Do we assume that gettext.sh it's there, or we provide
backfail mechanism? (for example I don't think that this file comes with
cygwin...)

I can translate in two ways:
----
echo $(gettext "hello world")
eval_gettext "hello world";echo
----

so, with eval_gettext (comes from gettext.sh) or from gettext binary
straight. 

eval_gettext helps a bit substituing shell variables:
# eval_gettext MSGID
# looks up the translation of MSGID and substitutes shell variables in the
# result.
eval_gettext () {
  gettext "$1" | (export PATH `envsubst --variables "$1"`; envsubst
"$1")
}

But we could do it without gettext.sh

Which method do you think that we should use?
With failback option?

Thanks,

-- 
Carles Pina i Estany		GPG id: 0x17756391
	http://pinux.info

Re: localization of Grub

[Robert Millan]

On Mon, Dec 08, 2008 at 12:33:08AM +0100, Carles Pina i Estany wrote:
> >   - gettextise the util tools, so they can be translated as normal programs.
> 
> Do you mean the strings that are generated in standard Linux user-space
> and written to grub.cfg?

I mean every command produced out of util/.  grub-mkconfig, grub-probe,
grub-setup, etc.

> What do you (everybody in the list) think about the text help that
> appears here and there (output of Grub -usually shell- commands?)
> Needed? Or these texts are good enough in English (like commands in the
> programming languages)

I think in long-term it would probably be a good thing to gettextize
everything; but short-term we should focus on the util/ part IMHO.

> This part is more clear (at least theoretically, of course :-) )
> I understand that all error messages, messages, etc. that Grub prints
> while Grub is executing should be translated.
> 
> My understanding is that we should have some module to handle some
> function (eg. _()), that would read some variable to choose to which
> language, if available, translate some string. Like gettext, as you
> said.

Yep, something like that.  Also, it needs to be registered in a function
pointer, so that kernel won't need it unconditionally.

> I understand that we/I should adapt some scripts using, for example, this
> guideline?:
> http://www.gnu.org/software/automake/manual/gettext/Preparing-Shell-Scripts.html

Probably.  I'm afraid I'm no l10n expert :-(

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Re: localization of Grub

[Robert Millan]

On Mon, Dec 08, 2008 at 03:33:49PM +0100, Carles Pina i Estany wrote:
> 
> Hello,
> 
> On Dec/08/2008, Carles Pina i Estany wrote:
> 
> > I understand that we/I should adapt some scripts using, for example, this
> > guideline?:
> > http://www.gnu.org/software/automake/manual/gettext/Preparing-Shell-Scripts.html
> > 
> > (I haven't tested now, I think that some years ago I did some tests with
> 
> They use gettext.sh file, in my Debian is in:
> /usr/bin/gettext.sh, in package gettext-base . Do you think that this is
> the best way? Do we assume that gettext.sh it's there, or we provide
> backfail mechanism? (for example I don't think that this file comes with
> cygwin...)
> 
> I can translate in two ways:
> ----
> echo $(gettext "hello world")
> eval_gettext "hello world";echo
> ----
> 
> so, with eval_gettext (comes from gettext.sh) or from gettext binary
> straight. 
> 
> eval_gettext helps a bit substituing shell variables:
> # eval_gettext MSGID
> # looks up the translation of MSGID and substitutes shell variables in the
> # result.
> eval_gettext () {
>   gettext "$1" | (export PATH `envsubst --variables "$1"`; envsubst
> "$1")
> }
> 
> But we could do it without gettext.sh
> 
> Which method do you think that we should use?
> With failback option?

I have no idea about this, and others in this list didn't reply.  I suggest
you ask Jordi Mallach (CCed).  He's quite knowledgeable about such things
and often hangs around here too ;-)

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Re: localization of Grub

[Carles Pina i Estany]

Hello,

On Dec/14/2008, Robert Millan wrote:
> On Mon, Dec 08, 2008 at 03:33:49PM +0100, Carles Pina i Estany wrote:

> > Which method do you think that we should use?
> > With failback option?
> 
> I have no idea about this, and others in this list didn't reply.  I suggest
> you ask Jordi Mallach (CCed).  He's quite knowledgeable about such things
> and often hangs around here too ;-)

I've resent the mail to Jordi...

Today morning I've implemented a .mo reader, based on this
specification:
http://www.gnu.org/software/gettext/manual/html_node/MO-Files.html#MO-Files

It's a very basic implementation, and not inside Grub but as a C
standalone program.

After having the the shell scripts localization I will port my .mo
reader to Grub.

Right now is not using the hash table, we save a bit of space in the .mo
files and in code (I haven't implemented, I don't think that Grub has so
much strings to need the hash table, specially when the documentation
says:
-----------
The size S of the hash table can be zero. In this case, the hash table
itself is not contained in the MO file. Some people might prefer this
because a precomputed hashing table takes disk space, and does not win
that much speed. The hash table contains indices to the sorted array of
strings in the MO file. Conflict resolution is done by double hashing.
The precise hashing algorithm used is fairly dependent on GNU gettext
code, and is not documented here. 
-----------

Happy new year,

-- 
Carles Pina i Estany		GPG id: 0x17756391
	http://pinux.info