Re: Fw: [Bugme-new] [Bug 4137] New: nls_cp936.c is not synchronized with M$'s translation table

Re: Fw: [Bugme-new] [Bug 4137] New: nls_cp936.c is not synchronized with M$'s translation table

[Andries Brouwer]

On Tue, Feb 01, 2005 at 01:14:39AM +0900, OGAWA Hirofumi wrote:
> Andries Brouwer <Andries.Brouwer@cwi.nl> writes:
> > On Sun, Jan 30, 2005 at 09:25:33PM -0800, Andrew Morton wrote:
> >
> >> The nls_cp936.c is not synchronized with MS's translation table, there are some
> >> characters have different code from the code in updated table of MS.
> >> For example, the unicode for character 0xB8A3 (GBK) should be U+798F, but the
> >> code is U+FA1B in nls_cp936.c.
> >
> > OK - regenerated the tables.
> 
> > @@ -1510,13 +1499,13 @@ static wchar_t c2u_FA[256] = {
> > -	0x2166,0x2167,0x2168,0x2169,0x0000,0xFFE4,0xFF07,0xFF02,/* 0x50-0x57 */
> > +	0x2166,0x2167,0x2168,0x2169,0xFFE2,0xFFE4,0xFF07,0xFF02,/* 0x50-0x57 */
> 
> Microsoft is using this, and some other vendor is using 0x00AC. The
> both of those char code is valid for fatfs...
> 
> Anyway, this patch looks good to me.

Let me cc the submitter of the bug report and linux-kernel.
Below a description of the changes in the four code pages 932, 936, 949, 950.
(They have been aligned to current Microsoft tables.)

Andries


nls_cp932.c:

0x0000 -> 0xFFE2	FULL WIDTH NOT SIGN

Was already present as 0x81CA and 0xEEF9.
Now also at 0xFA54.

nls_cp936.c:

0x2218 -> 0x00B0	RING OPERATOR -> DEGREE SIGN

CJK Compatibility ideographs replaced by corresponding ordinary symbol.

0xF900 -> 0x8C48
0xF901 -> 0x66F4
0xF902 -> 0x8ECA
0xF903 -> 0x8CC8
0xF904 -> 0x6ED1
0xF905 -> 0x4E32
0xF906 -> 0x53E5
0xF908 -> 0x9F9C
0xF909 -> 0x5951
0xF90A -> 0x91D1
0xF90B -> 0x5587
0xF90C -> 0x5948
0xF90D -> 0x61F6
0xF90E -> 0x7669
0xF90F -> 0x7F85
0xF910 -> 0x863F
0xF911 -> 0x87BA
0xF912 -> 0x88F8
0xF913 -> 0x908F
0xF915 -> 0x6D1B
0xF916 -> 0x70D9
0xF917 -> 0x73DE
0xF918 -> 0x843D
0xF919 -> 0x916A
0xF91A -> 0x99F1
0xF91B -> 0x4E82
0xF91C -> 0x5375
0xF91D -> 0x6B04
0xF91E -> 0x721B
0xF91F -> 0x862D
0xF920 -> 0x9E1E
0xF921 -> 0x5D50
0xF922 -> 0x6FEB
0xF923 -> 0x85CD
0xF924 -> 0x8964
0xF925 -> 0x62C9
0xF926 -> 0x81D8
0xF927 -> 0x881F
0xF928 -> 0x5ECA
0xF929 -> 0x6717
0xF92A -> 0x6D6A
0xF92B -> 0x72FC
0xF92D -> 0x4F86
0xF92E -> 0x51B7
0xF92F -> 0x52DE
0xF930 -> 0x64C4
0xF931 -> 0x6AD3
0xF932 -> 0x7210
0xF933 -> 0x76E7
0xF934 -> 0x8001
0xF935 -> 0x8606
0xF936 -> 0x865C
0xF937 -> 0x8DEF
0xF938 -> 0x9732
0xF939 -> 0x9B6F
0xF93A -> 0x9DFA
0xF93B -> 0x788C
0xF93C -> 0x797F
0xF93D -> 0x7DA0
0xF93E -> 0x83C9
0xF93F -> 0x9304
0xF940 -> 0x9E7F
0xF941 -> 0x8AD6
0xF942 -> 0x58DF
0xF943 -> 0x5F04
0xF944 -> 0x7C60
0xF945 -> 0x807E
0xF946 -> 0x7262
0xF947 -> 0x78CA
0xF948 -> 0x8CC2
0xF949 -> 0x96F7
0xF94A -> 0x58D8
0xF94B -> 0x5C62
0xF94C -> 0x6A13
0xF94D -> 0x6DDA
0xF94E -> 0x6F0F
0xF94F -> 0x7D2F
0xF950 -> 0x7E37
0xF951 -> 0x964B
0xF952 -> 0x52D2
0xF953 -> 0x808B
0xF954 -> 0x51DC
0xF955 -> 0x51CC
0xF956 -> 0x7A1C
0xF957 -> 0x7DBE
0xF958 -> 0x83F1
0xF959 -> 0x9675
0xF95A -> 0x8B80
0xF95B -> 0x62CF
0xF95D -> 0x8AFE
0xF95E -> 0x4E39
0xF960 -> 0x6012
0xF962 -> 0x7570
0xF963 -> 0x5317
0xF964 -> 0x78FB
0xF965 -> 0x4FBF
0xF966 -> 0x5FA9
0xF967 -> 0x4E0D
0xF968 -> 0x6CCC
0xF969 -> 0x6578
0xF96A -> 0x7D22
0xF96B -> 0x53C3
0xF96C -> 0x585E
0xF96D -> 0x7701
0xF96E -> 0x8449
0xF970 -> 0x6BBA
0xF971 -> 0x8FB0
0xF972 -> 0x6C88
0xF973 -> 0x62FE
0xF974 -> 0x82E5
0xF975 -> 0x63A0
0xF976 -> 0x7565
0xF977 -> 0x4EAE
0xF978 -> 0x5169
0xF97A -> 0x6881
0xF97B -> 0x7CE7
0xF97C -> 0x826F
0xF97D -> 0x8AD2
0xF97E -> 0x91CF
0xF97F -> 0x52F5
0xF980 -> 0x5442
0xF981 -> 0x5973
0xF982 -> 0x5EEC
0xF983 -> 0x65C5
0xF984 -> 0x6FFE
0xF985 -> 0x792A
0xF986 -> 0x95AD
0xF987 -> 0x9A6A
0xF988 -> 0x9E97
0xF989 -> 0x9ECE
0xF98A -> 0x529B
0xF98B -> 0x66C6
0xF98C -> 0x6B77
0xF98D -> 0x8F62
0xF98E -> 0x5E74
0xF98F -> 0x6190
0xF990 -> 0x6200
0xF991 -> 0x649A
0xF992 -> 0x6F23
0xF993 -> 0x7149
0xF994 -> 0x7489
0xF996 -> 0x7DF4
0xF997 -> 0x806F
0xF998 -> 0x8F26
0xF999 -> 0x84EE
0xF99A -> 0x9023
0xF99B -> 0x934A
0xF99C -> 0x5217
0xF99D -> 0x52A3
0xF99E -> 0x54BD
0xF99F -> 0x70C8
0xF9A0 -> 0x88C2
0xF9A1 -> 0x8AAA
0xF9A2 -> 0x5EC9
0xF9A3 -> 0x5FF5
0xF9A4 -> 0x637B
0xF9A5 -> 0x6BAE
0xF9A6 -> 0x7C3E
0xF9A7 -> 0x7375
0xF9A8 -> 0x4EE4
0xF9A9 -> 0x56F9
0xF9AA -> 0x5BE7
0xF9AB -> 0x5DBA
0xF9AC -> 0x601C
0xF9AD -> 0x73B2
0xF9AE -> 0x7469
0xF9AF -> 0x7F9A
0xF9B0 -> 0x8046
0xF9B1 -> 0x9234
0xF9B2 -> 0x96F6
0xF9B3 -> 0x9748
0xF9B4 -> 0x9818
0xF9B5 -> 0x4F8B
0xF9B6 -> 0x79AE
0xF9B7 -> 0x91B4
0xF9B8 -> 0x96B8
0xF9B9 -> 0x60E1
0xF9BA -> 0x4E86
0xF9BB -> 0x50DA
0xF9BC -> 0x5BEE
0xF9BD -> 0x5C3F
0xF9BE -> 0x6599
0xF9BF -> 0x6A02
0xF9C0 -> 0x71CE
0xF9C1 -> 0x7642
0xF9C2 -> 0x84FC
0xF9C3 -> 0x907C
0xF9C4 -> 0x9F8D
0xF9C5 -> 0x6688
0xF9C6 -> 0x962E
0xF9C7 -> 0x5289
0xF9C8 -> 0x677B
0xF9C9 -> 0x67F3
0xF9CA -> 0x6D41
0xF9CB -> 0x6E9C
0xF9CC -> 0x7409
0xF9CD -> 0x7559
0xF9CE -> 0x786B
0xF9CF -> 0x7D10
0xF9D0 -> 0x985E
0xF9D1 -> 0x516D
0xF9D2 -> 0x622E
0xF9D3 -> 0x9678
0xF9D4 -> 0x502B
0xF9D5 -> 0x5D19
0xF9D6 -> 0x6DEA
0xF9D7 -> 0x8F2A
0xF9D8 -> 0x5F8B
0xF9D9 -> 0x6144
0xF9DA -> 0x6817
0xF9DB -> 0x7387
0xF9DC -> 0x9686
0xF9DD -> 0x5229
0xF9DE -> 0x540F
0xF9DF -> 0x5C65
0xF9E0 -> 0x6613
0xF9E1 -> 0x674E
0xF9E2 -> 0x68A8
0xF9E3 -> 0x6CE5
0xF9E4 -> 0x7406
0xF9E5 -> 0x75E2
0xF9E6 -> 0x7F79
0xF9E8 -> 0x88E1
0xF9E9 -> 0x91CC
0xF9EA -> 0x96E2
0xF9EB -> 0x533F
0xF9EC -> 0x6EBA
0xF9ED -> 0x541D
0xF9EE -> 0x71D0
0xF9EF -> 0x7498
0xF9F0 -> 0x85FA
0xF9F2 -> 0x9C57
0xF9F3 -> 0x9E9F
0xF9F4 -> 0x6797
0xF9F5 -> 0x6DCB
0xF9F6 -> 0x81E8
0xF9F7 -> 0x7ACB
0xF9F8 -> 0x7B20
0xF9F9 -> 0x7C92
0xF9FA -> 0x72C0
0xF9FB -> 0x7099
0xF9FC -> 0x8B58
0xF9FD -> 0x4EC0
0xF9FE -> 0x8336
0xF9FF -> 0x523A
0xFA00 -> 0x5207
0xFA01 -> 0x5EA6
0xFA02 -> 0x62D3
0xFA03 -> 0x7CD6
0xFA04 -> 0x5B85
0xFA05 -> 0x6D1E
0xFA06 -> 0x66B4
0xFA07 -> 0x8F3B
0xFA08 -> 0x884C
0xFA09 -> 0x964D
0xFA0A -> 0x898B
0xFA0B -> 0x5ED3
0xFA10 -> 0x585A
0xFA12 -> 0x6674
0xFA15 -> 0x51DE
0xFA16 -> 0x8C6C expected: 732A, but that code already occurs elsewhere
0xFA17 -> 0x76CA
0xFA19 -> 0x795E
0xFA1A -> 0x7965
0xFA1B -> 0x798F
0xFA1C -> 0x9756
0xFA1D -> 0x7CBE
0xFA1E -> 0x7FBD
0xFA22 -> 0x8AF8
0xFA25 -> 0x9038
0xFA26 -> 0x90FD
0xFA2A -> 0x98EF
0xFA2B -> 0x98FC
0xFA2C -> 0x9928
0xFA2D -> 0x9DB4

nls_cp949.c;

New:

0x0000 -> 0x20AC	EURO SIGN
0x0000 -> 0x00AE	REGISTERED SIGN

Halfwidth Hangul replaced by corresponding ordinary symbols.

0xFFA1...0xFFBE (30), 0xFFC2...0xFFC7 (6), 0xFFCA-0xFFCF (6),
0xFFD2...0xFFD7 (6), 0xFFDA-0xFFDC (3), 0xFFA0 (1)
->
0x3131...0x3164 (52)

More CJK Compatibility ideographs replaced.

0xFA0C -> 0x5140
0xFA10 -> 0x585A
0xFA12 -> 0x6674
0xFA15 -> 0x51DE
0xFA17 -> 0x76CA
0xFA19 -> 0x795E
0xFA1A -> 0x7965
0xFA1B -> 0x798F
0xFA1C -> 0x9756
0xFA1D -> 0x7CBE
0xFA1E -> 0x7FBD
0xFA22 -> 0x8AF8
0xFA25 -> 0x9038
0xFA26 -> 0x90FD
0xFA2A -> 0x98EF
0xFA2B -> 0x98FC
0xFA2C -> 0x9928
0xFA2D -> 0x9DB4

nls_cp950.c:

New

0x0000 -> 0x20AC
0x0000 -> 0x2550
0x0000 -> 0x255E
0x0000 -> 0x2561
0x0000 -> 0x256A
0x0000 -> 0x256D
0x0000 -> 0x256E
0x0000 -> 0x256F
0x0000 -> 0x2570
0x0000 -> 0x5341
0x0000 -> 0x5345

Misc corrections

0x0305 -> 0x00AF
0x2024 -> 0x02D9
0x2218 -> 0x00B0
0x2263 -> 0x2261
0x2609 -> 0x2299
0x2641 -> 0x2295

CJK Compatibility ideographs replaced by corresponding ordinary symbol.
(Same changes as above on 0xF900-0xFA2D.)

Some fullwidth forms replaced by other forms

0xFF02 -> 0x201D
0xFF07 -> 0x2019
0xFF3B -> 0x3014
0xFF3D -> 0x3015
0xFF3E -> 0xFE3F
0xFF40 -> 0x2018
0xFF64 -> 0xFE51