Unicode policy

Unicode policy

[Philipp Gühring]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

I have some question regarding the Unicode policy of filenames in filesystem, 
especially ReiserFS. 

Are filenames stored in UTF-8?
Should they be stored in UTF-8?
Is it a question of the filesystem?
Is it the applications job to interpret it?
Or is the filesystem encoding independent?
Are there migration plans to Unicode?

Any feedback is very appreciated.

Many greetings,
- -- 
~ Philipp Gühring              p.guehring@futureware.at
~ http://www.livingxml.net/       ICQ UIN: 6588261
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE+sT9elqQ+F+0wB3oRAkm+AJwJbu/dNttqDRA5zPfBmAASndS1TgCgjQJW
Q1hj0U8sHOKgJ7Tu0ooWyOQ=
=PvVE
-----END PGP SIGNATURE-----

Re: Unicode policy

[Oleg Drokin]

Hello!

On Thu, May 01, 2003 at 05:38:04PM +0200, Philipp G?hring wrote:

> I have some question regarding the Unicode policy of filenames in filesystem, 
> especially ReiserFS. 
> Are filenames stored in UTF-8?

No. filenames are stored "as is". If you write those as unicode, you get unicode.

There are several restrictions on filenames in Unix/Linux:
You cannot use 0-byte and '/'-byte as part of filename.

> Should they be stored in UTF-8?

You decide how do you want to store your filenames.

> Is it a question of the filesystem?

Some filesystems do it, but I doubt FS should do it.

> Is it the applications job to interpret it?

It is debatable. Kernel/VFS/some library might do this as well.

> Or is the filesystem encoding independent?

Some filesystems are encoding-dependent and some are not (because they do not know anything about encoding at all).

> Are there migration plans to Unicode?

Migration of what? Migration of reiserfs to unicode is not planned.
Same for reiser4. You can write whatever plugin to store filenames in any way
you might imagine in reiser4.

Bye,
    Oleg

Re: Unicode policy

[Hans Reiser]

It is a VFS issue.  We just conform.  VFS can't handle / and null being 
in a filename, so Unicode can't work in Linux.  If you get VFS to handle 
Unicode, I'll make ReiserFS work.  Linus is the one to convince, not us.

Best,

Hans

Oleg Drokin wrote:

>Hello!
>
>On Thu, May 01, 2003 at 05:38:04PM +0200, Philipp G?hring wrote:
>
>  
>
>>I have some question regarding the Unicode policy of filenames in filesystem, 
>>especially ReiserFS. 
>>Are filenames stored in UTF-8?
>>    
>>
>
>No. filenames are stored "as is". If you write those as unicode, you get unicode.
>
>There are several restrictions on filenames in Unix/Linux:
>You cannot use 0-byte and '/'-byte as part of filename.
>
>  
>
>>Should they be stored in UTF-8?
>>    
>>
>
>You decide how do you want to store your filenames.
>
>  
>
>>Is it a question of the filesystem?
>>    
>>
>
>Some filesystems do it, but I doubt FS should do it.
>
>  
>
>>Is it the applications job to interpret it?
>>    
>>
>
>It is debatable. Kernel/VFS/some library might do this as well.
>
>  
>
>>Or is the filesystem encoding independent?
>>    
>>
>
>Some filesystems are encoding-dependent and some are not (because they do not know anything about encoding at all).
>
>  
>
>>Are there migration plans to Unicode?
>>    
>>
>
>Migration of what? Migration of reiserfs to unicode is not planned.
>Same for reiser4. You can write whatever plugin to store filenames in any way
>you might imagine in reiser4.
>
>Bye,
>    Oleg
>
>
>  
>


-- 
Hans

Re: Unicode policy

[Philipp Gühring]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Am Freitag, 2. Mai 2003 02:55 schrieben Sie:
> It is a VFS issue.  We just conform.  VFS can't handle / and null being
> in a filename, so Unicode can't work in Linux.  If you get VFS to handle
> Unicode, I'll make ReiserFS work.  Linus is the one to convince, not us.

Hi Hans,

UTF-8 (is also Unicode) has no nulls in it, and does not 
generate '/' or other problematic charakters, and is actually working on 
Reiserfs.

The problem with a migration to Unicode is that we either have to switch all 
applications at once, or have the information in the filesystem, in which 
encoding the filenames are ...

But you are right, it's a VFS issue ...

Many greetings,
- -- 
~ Philipp Gühring              p.guehring@futureware.at
~ http://www.livingxml.net/       ICQ UIN: 6588261
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE+scdqlqQ+F+0wB3oRAn8gAKCBNzPvsRsaxJT96fCEGvO6MXeFKgCdEXY1
xmMW0hHJZTkHM7GhQj0FcdI=
=U423
-----END PGP SIGNATURE-----

Re: Unicode policy

[Hans Reiser]

Philipp Gühring wrote:

>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>Am Freitag, 2. Mai 2003 02:55 schrieben Sie:
>  
>
>>It is a VFS issue.  We just conform.  VFS can't handle / and null being
>>in a filename, so Unicode can't work in Linux.  If you get VFS to handle
>>Unicode, I'll make ReiserFS work.  Linus is the one to convince, not us.
>>    
>>
>
>Hi Hans,
>
>UTF-8 (is also Unicode) has no nulls in it, and does not 
>generate '/' or other problematic charakters, and is actually working on 
>Reiserfs.
>
>  
>
Well, in that case, everything is done, VFS is fine as is, just start 
using UTF-8.  I am not sure who decides how to render UTF-8, but that 
would presumably be outside the kernel.

>The problem with a migration to Unicode is that we either have to switch all 
>applications at once, or have the information in the filesystem, in which 
>encoding the filenames are ...
>
>But you are right, it's a VFS issue ...
>
>Many greetings,
>- -- 
>~ Philipp Gühring              p.guehring@futureware.at
>~ http://www.livingxml.net/       ICQ UIN: 6588261
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v1.0.6 (GNU/Linux)
>Comment: For info see http://www.gnupg.org
>
>iD8DBQE+scdqlqQ+F+0wB3oRAn8gAKCBNzPvsRsaxJT96fCEGvO6MXeFKgCdEXY1
>xmMW0hHJZTkHM7GhQj0FcdI=
>=U423
>-----END PGP SIGNATURE-----
>
>
>  
>


-- 
Hans

Re: Unicode policy

[Anders Widman]

>>
>>UTF-8 (is also Unicode) has no nulls in it, and does not 
>>generate '/' or other problematic charakters, and is actually working on 
>>Reiserfs.
>>
> Well, in that case, everything is done, VFS is fine as is, just start 
> using UTF-8.  I am not sure who decides how to render UTF-8, but that 
> would presumably be outside the kernel.

It  is  up  to  the application to treat the names as they whish.. For
example  bash,  mc  and  gnome programs treat differently. At least in
most  Gnome  2 programs UTF-8 works mostly :) However I do not know if
UTF-16/32 works.


--------
PGP public key: https://tnonline.net/secure/pgp_key.txt

Re: Unicode policy

[Yury Umanets]

Philipp Gühring wrote:

>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>Am Freitag, 2. Mai 2003 02:55 schrieben Sie:
>  
>
>>It is a VFS issue.  We just conform.  VFS can't handle / and null being
>>in a filename, so Unicode can't work in Linux.  If you get VFS to handle
>>Unicode, I'll make ReiserFS work.  Linus is the one to convince, not us.
>>    
>>
>
>Hi Hans,
>
>UTF-8 (is also Unicode) has no nulls in it, and does not 
>generate '/' or other problematic charakters, and is actually working on 
>Reiserfs.
>
UTF-8 only uses two bytes for non latin characters (they cannot be fit 
one byte) and one byte for latin ones. In the case of latin characters, 
codes to be used will be the same as in ISO-8859-1, that is latin1.

>
>The problem with a migration to Unicode is that we either have to switch all 
>applications at once, or have the information in the filesystem, in which 
>encoding the filenames are ...
>
>But you are right, it's a VFS issue ...
>
>Many greetings,
>- -- 
>~ Philipp Gühring              p.guehring@futureware.at
>~ http://www.livingxml.net/       ICQ UIN: 6588261
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v1.0.6 (GNU/Linux)
>Comment: For info see http://www.gnupg.org
>
>iD8DBQE+scdqlqQ+F+0wB3oRAn8gAKCBNzPvsRsaxJT96fCEGvO6MXeFKgCdEXY1
>xmMW0hHJZTkHM7GhQj0FcdI=
>=U423
>-----END PGP SIGNATURE-----
>
>
>  
>


-- 
Yury Umanets
"We're flying high, we're watching the world passes by..."