Re: Correct management of .po files for translation

["Daniel P. Berrange" <berrange@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 2624 bytes --]

On Wed, Apr 04, 2007 at 05:31:16PM +0100, Ewan Mellor wrote:
> On Wed, Apr 04, 2007 at 05:02:23PM +0100, Daniel P. Berrange wrote:
> 
> > Basically the code is abusing the gettext translation service to do both the
> > constant -> string conversion & the translation in one go. The normal way to
> > do the constant -> string conversion is to have a statically declared hash
> > table in the application itself. This contains the master english strings
> > annotated with  N_(...string..).  The xgettext program will then generate
> > the .pot files automatically in the correct format.
> > 
> > I'm attaching an example to show how this ought to work in the Xen case.
> > The makefiles could be hooked up to automatically run
> > 
> >   xgettext --keyword=N_ -o xen-xm.pot tools/python/xen/xm/XenAPI.py
> 
> Sounds good to me, Daniel, thanks.  Could you write a patch to hook up the
> Makefiles as well?  I reckon we'll be able to sneak this one in for 3.0.5 if
> you did.

I'm attaching a patch which hooks up the tools/python/Makefile to update
the master xen-xm.pot file with new translatable text on every biuld, and
to merge these into any translations. The install rule was tweaked to deal
with the different build process.

There is no actual language translation in the patch - when adding new 
translations simply 

   cd xen/xm/messages
   cp xen-xm.pot fr_FR.po
   vi fr_FR.po
    ....

Also edit tools/python/Makefile to add the language code 'fr_FR' to the
LINGUAS makefile variable.

NB, since I don't have a current xen-unstable.hg kernel build operational
on my dev hardware currently I've not yet been able to test the correct
operational of the XenAPI.py changes in this patch. Hopefully I'll be able
to get a working kernel on this machine soon

 a/tools/python/xen/xm/messages/en/xen-xm.po |   69 ----------------------------
 b/tools/python/remove-potcdate.sed          |   19 +++++++
 b/tools/python/xen/xm/messages/xen-xm.pot   |   63 +++++++++++++++++++++++++
 config/StdGNU.mk                            |    1 
 tools/python/Makefile                       |   68 ++++++++++++++++++++++-----
 tools/python/xen/xm/XenAPI.py               |   18 ++++++-
 7 files changed, 218 insertions(+), 83 deletions(-)


   Signed-off-by: Daniel P. Berrange <berrange@redhat.com>

Regards,
Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

[-- Attachment #2: xen-gettext.patch --]
[-- Type: text/plain, Size: 10228 bytes --]

diff -r 2d25db73ddc1 config/StdGNU.mk
--- a/config/StdGNU.mk	Wed Apr 04 10:42:58 2007 +0100
+++ b/config/StdGNU.mk	Thu Apr 05 15:49:50 2007 -0400
@@ -10,6 +10,7 @@ OBJDUMP    = $(CROSS_COMPILE)objdump
 OBJDUMP    = $(CROSS_COMPILE)objdump
 
 MSGFMT     = msgfmt
+MSGMERGE   = msgmerge
 
 INSTALL      = install
 INSTALL_DIR  = $(INSTALL) -d -m0755 -p
diff -r 2d25db73ddc1 tools/python/Makefile
--- a/tools/python/Makefile	Wed Apr 04 10:42:58 2007 +0100
+++ b/tools/python/Makefile	Thu Apr 05 17:44:15 2007 -0400
@@ -4,17 +4,54 @@ include $(XEN_ROOT)/tools/Rules.mk
 .PHONY: all
 all: build
 
-.PHONY: build
-build:
+# For each new supported translation, add its name here, eg 'fr_FR'
+# to cause the .po file to be built & installed, eg
+LINGUAS :=
+POPACKAGE := xen-xm
+PODIR := xen/xm/messages
+POTFILE := $(PODIR)/xen-xm.pot
+I18NSRCFILES = $(shell find xen/xm/ -name '*.py')
+CATALOGS = $(patsubst %,xen/xm/messages/%.mo,$(LINGUAS))
+NLSDIR = /usr/share/locale
+
+.PHONY: build buildpy
+buildpy:
 	CC="$(CC)" CFLAGS="$(CFLAGS)" python setup.py build
-	if which $(MSGFMT) >/dev/null ; then \
-          for file in `cd ./xen/xm; find messages -name xen-xm.po`; do \
-            dest=`echo "build/$$file" | \
-                  sed -e 's#xen-xm.po#LC_MESSAGES/xen-xm.mo#'`; \
-            mkdir -p `dirname "$$dest"`; \
-            $(MSGFMT) -c -o "$$dest" "xen/xm/$$file"; \
-          done; \
-        fi
+
+build: buildpy refresh-pot refresh-po $(CATALOGS)
+
+# NB we take care to only update the .pot file it strings have
+# actually changed. This is complicated by the embedded date
+# string, hence the sed black magic. This avoids the expensive
+# re-generation of .po files on every single build
+refresh-pot: $(I18NSRCFILES)
+	xgettext --default-domain=$(POPACAKGE) \
+		--keyword=N_ \
+		--keyword=_ \
+		-o $(POTFILE)-tmp \
+		$(I18NSRCFILES)
+	sed -f remove-potcdate.sed < $(POTFILE) > $(POTFILE)-1
+	sed -f remove-potcdate.sed < $(POTFILE)-tmp > $(POTFILE)-2
+	if cmp -s $(POTFILE)-1 $(POTFILE)-2; then \
+		rm -f $(POTFILE)-tmp $(POTFILE)-1 $(POTFILE)-2; \
+	else \
+		mv $(POTFILE)-tmp $(POTFILE); \
+                rm -f $(POTFILE)-1 $(POTFILE)-2; \
+	fi
+
+refresh-po: $(POTFILE)
+	for l in $(LINGUAS); do \
+		if $(MSGMERGE) $(PODIR)/$$l.po $(POTFILE) > $(PODIR)/$$l-tmp ; then \
+			mv -f $(PODIR)/$$l-tmp $(PODIR)/$$l.po ; \
+			echo "$(MSGMERGE) of $$l.po succeeded" ; \
+		else \
+			echo "$(MSGMERGE) of $$l.po failed" ; \
+			rm -f $(PODIR)/$$l-tmp ; \
+		fi \
+	done
+
+%.mo: %.po
+	$(MSGFMT) -c -o $@ $<
 
 .PHONY: install
 ifndef XEN_PYTHON_NATIVE_INSTALL
@@ -28,8 +65,13 @@ endif
 
 install-messages: all
 	if which $(MSGFMT) >/dev/null ; then \
-	  mkdir -p "$(DESTDIR)/usr/share/locale"; \
-	  cp -R build/messages/* "$(DESTDIR)/usr/share/locale/"; \
+		mkdir -p $(DESTDIR)$(NLSDIR); \
+		for l in $(LINGUAS); do \
+			$(INSTALL) -m 755 -d $(DESTDIR)$(NLSDIR)/$$l; \
+			$(INSTALL) -m 755 -d $(DESTDIR)$(NLSDIR)/$$l/LC_MESSAGES; \
+			$(INSTALL) -m 644 $(PODIR)/$$l.mo \
+				$(DESTDIR)$(NLSDIR)/$$l/LC_MESSAGES/$(POPACKAGE).mo; \
+		done ; \
 	fi
 
 .PHONY: test
@@ -38,4 +80,4 @@ test:
 
 .PHONY: clean
 clean:
-	rm -rf build *.pyc *.pyo *.o *.a *~
+	rm -rf build *.pyc *.pyo *.o *.a *~ $(CATALOGS)
diff -r 2d25db73ddc1 tools/python/remove-potcdate.sed
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/tools/python/remove-potcdate.sed	Thu Apr 05 15:38:30 2007 -0400
@@ -0,0 +1,19 @@
+# Sed script that remove the POT-Creation-Date line in the header entry
+# from a POT file.
+#
+# The distinction between the first and the following occurrences of the
+# pattern is achieved by looking at the hold space.
+/^"POT-Creation-Date: .*"$/{
+x
+# Test if the hold space is empty.
+s/P/P/
+ta
+# Yes it was empty. First occurrence. Remove the line.
+g
+d
+bb
+:a
+# The hold space was nonempty. Following occurrences. Do nothing.
+x
+:b
+}
diff -r 2d25db73ddc1 tools/python/xen/xm/XenAPI.py
--- a/tools/python/xen/xm/XenAPI.py	Wed Apr 04 10:42:58 2007 +0100
+++ b/tools/python/xen/xm/XenAPI.py	Thu Apr 05 14:17:11 2007 -0400
@@ -49,6 +49,22 @@ import xmlrpclib
 
 import xen.util.xmlrpcclient as xmlrpcclient
 
+def gettext_noop(str):
+    return str
+
+N_ = gettext_noop
+
+errormap = {
+    "INTERNAL_ERROR": N_("Internal error: %(1)s."),
+    "MAP_DUPLICATE_KEY": N_("This map already contains %(1)s -> %(2)s."),
+    "MESSAGE_METHOD_UNKNOWN": N_("The method %(1)s is unsupported."),
+    "MESSAGE_PARAMETER_COUNT_MISMATCH": N_("The method %(1)s takes %(2)s argument(s) (%(3)s given)."),
+    "SESSION_AUTHENTICATION_FAILED": N_("Permission denied."),
+    "VALUE_NOT_SUPPORTED": N_("Value \"%(2)s\" for %(1)s is not supported by this server.  The server said \"%(3)s\"."),
+    "HANDLE_INVALID": N_("The %(1)s handle %(2)s is invalid."),
+    "OPERATION_NOT_ALLOWED": N_("You attempted an operation that was not allowed."),
+    "NETWORK_ALREADY_CONNECTED": N_("The network you specified already has a PIF attached to it, and so another one may not be attached."),
+    }
 
 translation = gettext.translation('xen-xm', fallback = True)
 
@@ -68,7 +84,7 @@ class Failure(Exception):
 
     def __str__(self):
         try:
-            return translation.ugettext(self.details[0]) % self._details_map()
+            return translation.ugettext(errormap[self.details[0]]) % self._details_map()
         except TypeError, exn:
             return "Message database broken: %s.\nXen-API failure: %s" % \
                    (exn, str(self.details))
diff -r 2d25db73ddc1 tools/python/xen/xm/messages/en/xen-xm.po
--- a/tools/python/xen/xm/messages/en/xen-xm.po	Wed Apr 04 10:42:58 2007 +0100
+++ /dev/null	Thu Jan 01 00:00:00 1970 +0000
@@ -1,69 +0,0 @@
-# ============================================================================
-# This library is free software; you can redistribute it and/or
-# modify it under the terms of version 2.1 of the GNU Lesser General Public
-# License as published by the Free Software Foundation.
-# 
-# This library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-# Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public
-# License along with this library; if not, write to the Free Software
-# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
-# ============================================================================
-# Copyright (c) 2006-2007 XenSource Inc.
-# ============================================================================
-# 
-#
-msgid ""
-msgstr ""
-"Project-Id-Version: Xen-xm 3.0\n"
-"PO-Revision-Date: 2007-03-29 16:13+0100\n"
-"Last-Translator: Ewan Mellor <ewan@xensource.com>\n"
-"Language-Team: xen-devel <xen-devel@lists.xensource.com>\n"
-"MIME-Version: 1.0\n"
-"Content-Type: text/plain; charset=ASCII\n"
-"Content-Transfer-Encoding: 8bit\n"
-
-msgid "INTERNAL_ERROR"
-msgstr "Internal error: %(1)s."
-
-msgid "MAP_DUPLICATE_KEY"
-msgstr "This map already contains %(1)s -> %(2)s."
-
-msgid "MESSAGE_METHOD_UNKNOWN"
-msgstr "The method %(1)s is unsupported."
-
-msgid "MESSAGE_PARAMETER_COUNT_MISMATCH"
-msgstr "The method %(1)s takes %(2)s argument(s) (%(3)s given)."
-
-msgid "SESSION_AUTHENTICATION_FAILED"
-msgstr "Permission denied."
-
-msgid "VALUE_NOT_SUPPORTED"
-msgstr "Value \"%(2)s\" for %(1)s is not supported by this server.  The server said \"%(3)s\"."
-
-msgid "HANDLE_INVALID"
-msgstr "The %(1)s handle %(2)s is invalid."
-
-msgid "OPERATION_NOT_ALLOWED"
-msgstr "You attempted an operation that was not allowed."
-
-msgid "NETWORK_ALREADY_CONNECTED"
-msgstr "The network you specified already has a PIF attached to it, and so another one may not be attached."
-
-msgid "PIF_IS_PHYSICAL"
-msgstr "The PIF %(1)s corresponds to a physical interface, and so may not be destroyed."
-
-msgid "VLAN_TAG_INVALID"
-msgstr "The VLAN tag you gave (%(1)s) is invalid -- it must be between 0 and 4095."
-
-msgid "VM_BAD_POWER_STATE"
-msgstr "The VM must be %(2)s to perform the requested operation (it is currently %(3)s)."
-
-msgid "VM_HVM_REQUIRED"
-msgstr "HVM guest support is unavailable: is VT/AMD-V supported by your CPU and enabled in your BIOS?"
-
-msgid "SESSION_NOT_REGISTERED"
-msgstr "This session is not registered to receive events.  You must call event.register before event.next.  (Session handle is %(1)s.)"
diff -r 2d25db73ddc1 tools/python/xen/xm/messages/xen-xm.pot
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/tools/python/xen/xm/messages/xen-xm.pot	Thu Apr 05 14:17:21 2007 -0400
@@ -0,0 +1,63 @@
+# SOME DESCRIPTIVE TITLE.
+# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
+# This file is distributed under the same license as the PACKAGE package.
+# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
+#
+#, fuzzy
+msgid ""
+msgstr ""
+"Project-Id-Version: PACKAGE VERSION\n"
+"Report-Msgid-Bugs-To: \n"
+"POT-Creation-Date: 2007-04-05 14:17-0400\n"
+"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
+"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
+"Language-Team: LANGUAGE <LL@li.org>\n"
+"MIME-Version: 1.0\n"
+"Content-Type: text/plain; charset=CHARSET\n"
+"Content-Transfer-Encoding: 8bit\n"
+
+#: xen/xm/XenAPI.py:58
+#, python-format
+msgid "Internal error: %(1)s."
+msgstr ""
+
+#: xen/xm/XenAPI.py:59
+#, python-format
+msgid "This map already contains %(1)s -> %(2)s."
+msgstr ""
+
+#: xen/xm/XenAPI.py:60
+#, python-format
+msgid "The method %(1)s is unsupported."
+msgstr ""
+
+#: xen/xm/XenAPI.py:61
+#, python-format
+msgid "The method %(1)s takes %(2)s argument(s) (%(3)s given)."
+msgstr ""
+
+#: xen/xm/XenAPI.py:62
+msgid "Permission denied."
+msgstr ""
+
+#: xen/xm/XenAPI.py:63
+#, python-format
+msgid ""
+"Value \"%(2)s\" for %(1)s is not supported by this server.  The server said "
+"\"%(3)s\"."
+msgstr ""
+
+#: xen/xm/XenAPI.py:64
+#, python-format
+msgid "The %(1)s handle %(2)s is invalid."
+msgstr ""
+
+#: xen/xm/XenAPI.py:65
+msgid "You attempted an operation that was not allowed."
+msgstr ""
+
+#: xen/xm/XenAPI.py:66
+msgid ""
+"The network you specified already has a PIF attached to it, and so another "
+"one may not be attached."
+msgstr ""

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel