Re: UTF-8 practically vs. theoretically in the VFS API (was: Re: JFS default behavior)

[Marc Lehmann <pcg@schmorp.de>]

On Tue, Feb 17, 2004 at 01:04:14PM -0800, Linus Torvalds <torvalds@osdl.org> wrote:
> Admittedly you might need up to six octets for the worst case, but hey, 
> since you only need one for the most common case (by _far_), who cares?

Beign a fan of UTF-8, I still have to remark that this is a rather imperialistic view that only
happens to work in many western countries.

It starts to fail in greece, russia and asian countries, where text size
goes up by a factor of 1.5 .. 3.

This was _one_ of the major obstacles that utf-8 had to overcome in asian
countries.

Personally, I think that's not a big problem (memory for text storage is
cheap etc.. :), but I am living in a iso-8859-1 world with only occasional
voyages elsewhere.

> And with the same UTF-8 encoding, you could some day encode UCS-8 too if
> the idiotic standards bodies some day decide that 4 billion characters 
> isn't enough because of all the in-fighting. 

Four billion glyphs will be not be reached, of course, but it's not
impossible that some codeset space inflation will happen due to the
introduction of extra planes for strange purposes.

> Of course, since you like UCS-4, you don't care about backwards 
> compatibility. 

While UCS-2 is obviously useless, UCS-4 is useful in rare cases where you
either need fixed character sizes or the inflation to 5 or 6 byte values
becomes a problem (which should be never).

Using UCS-4 for filenames is just evil (of course :)

UTF-8 was invented for the purpose of mapping unicode to filenames, and
it certainly is the most sane encoding so far, since it doesn't share the
"artificial" limitations to 16, 21 or 32 bits that other unicode encodings
have.

-- 
      -----==-                                             |
      ----==-- _                                           |
      ---==---(_)__  __ ____  __       Marc Lehmann      +--
      --==---/ / _ \/ // /\ \/ /       pcg@goof.com      |e|
      -=====/_/_//_/\_,_/ /_/\_\       XX11-RIPE         --+
    The choice of a GNU generation                       |
                                                         |