Re: Git-mediawiki : Encoding problems in perl

[Jakub Narebski <jnareb@gmail.com>]

Jérémie NIKAES <jeremie.nikaes@gmail.com> writes:

> While working on the git-mediawiki project[1], we ran into some
> problems regarding utf8 encoding of files. Most of them have been
> solved, however, one is still pretty annoying.
> Let me illustrate it :
> 
> I want to edit a page on mediawiki using the API, with a very simple example :
> 
> my $mw = MediaWiki::API->new();
> $mw->edit( {
>         action => 'edit',
>         title => 'Main_page',
>         text => 'été',
> } ) ;
> 
> But, when I look at the page on mediawiki, I see weird characters : été.

Take a look at

  http://stackoverflow.com/questions/6162484/why-does-modern-perl-avoid-utf-8-by-default

especially accepted answer.


In short (I don't agree with everything there, and not everything is
needed for all but extremal Unicode usage): if your script is written
using UTF-8 like in above examples, use

  use utf8;

If this is simplification, and this text comes from other file or is
result of output of some command, use

  use utf8::all;

or take a look what it does and put relevant parts in your script.
 
> I tried text => encode_utf8('été') with no success.
> 
> This makes pushing changes from git to mediawiki buggy since pulling a
> file with accentuated characters and pushing it right after changes
> things on the wiki.
> 
> While googling (a lot), I found that utf8 was pretty tricky in perl...
> The only thing that seems to solve things is a simple addition of 'use
> encoding utf8' at the top of our script.
> However
> A) Adding this line requires that I remove 'use strict;'

  use encoding ':utf8';

or

  use encoding 'utf8';

> B) I found some information about this pragma encoding and it seems to
> be unadvised to use it

-- 
Jakub Narebski
Poland
ShadeHawk on #git