Re: [Patch] Support UTF-8 scripts

["\"Martin v. Löwis\"" <martin@v.loewis.de>]

Martin Mares wrote:
> This makes no sense. For a script, the shell does not care about the encoding
> at all.

I'm not (only) talking about /bin/sh. I'm primarily talking about
/usr/bin/python, /usr/bin/perl, and /usr/bin/wish. In all these
languages, the interpreter *does* care about the encoding.

1. In Python, the syntax

   u"some data"

   denotes a Unicode literal (stored internally either in UCS-2 or
   UCS-4); the literals are converted from the source encoding to
   the internal representation. This requires knowledge of the source
   encoding.

2. In Tcl, all strings are internally represented in UTF-8, and
   converted from the source encoding (which currently is inferred
   from the locale of the process executing the script).

3. In Perl, 'use utf8' declares that the encoding of the script is
   UTF-8, meaning that non-ASCII can be used in string literals,
   identifiers, and regular expressions.

> Also, currently, people use zillions of encodings, most of which have no
> signature, so introducing a signature for UTF-8 does not win anything.

This specific patch does win something: it allows to executed scripts
which start with <utf8 signature>#!

This is useful e.g. for Python, which recognizes the UTF-8 signature
as declaring the source encoding of the Python module to be UTF-8.

> In the future, most people will probably use only UTF-8, so the signature
> carries no information.

In the future, the signature *will* carry no information. But the future
is, well, in the future.

I just can't understand why (some) people are so opposed to this patch.
It is a really trivial, straight-forward change. It introduces no
policy, just a feature: you can put the UTF-8 signature in your script
file, if you want to (and your scripting language supports it). By
no means it forces you to put the UTF-8 signature in your all script
files, let alone all your text files.

Regards,
Martin