character-problems

character-problems

[Thomas Creutz]

Hi List,

I use samba server on my webserver and mount the share with a linux
client to backup the files remote. Problem is, that I cannot access all
files with linux cifs client and secondly filenames are changed, so that
the backup (when possible to access the files) is not well done.

On the server output of ls -lah is:
insgesamt 400K
drwxr-xr-x  2 web178 client14 4,0K Mai 28  2008 .
drwxrwxrwx 13 web178 client14 4,0K Okt 17  2013 ..
-rw-r--r--  1 web178 client14  59K Mai 28  2008 JulianKiendlGitarre.jpg
-rw-r--r--  1 web178 client14  79K Mai 28  2008 Kost?me1.jpg
-rw-r--r--  1 web178 client14  51K Mai 28  2008 Kost?me2.jpg
-rw-r--r--  1 web178 client14  55K Mai 28  2008 MuellerReeb.jpg
-rw-r--r--  1 web178 client14  42K Mai 28  2008 Partyraum.jpg
-rw-r--r--  1 web178 client14  60K Mai 28  2008 Schlegel.jpg
-rw-r--r--  1 web178 client14  38K Mai 28  2008 Tanzen.jpg

on the client side:
insgesamt 392K
drwxr-xr-x  2 1124 1051   0 Mai 28  2008 .
drwxrwxrwx 13 1124 1051   0 Okt 17  2013 ..
-rw-r--r--  1 1124 1051 59K Mai 28  2008 JulianKiendlGitarre.jpg
-rw-r--r--  1 1124 1051 79K Mai 28  2008 Kost_me1.jpg
-rw-r--r--  1 1124 1051 51K Mai 28  2008 Kost_me2.jpg
-rw-r--r--  1 1124 1051 55K Mai 28  2008 MuellerReeb.jpg
-rw-r--r--  1 1124 1051 42K Mai 28  2008 Partyraum.jpg
-rw-r--r--  1 1124 1051 60K Mai 28  2008 Schlegel.jpg
-rw-r--r--  1 1124 1051 38K Mai 28  2008 Tanzen.jpg

LANG=C cp Kost_me1.jpg /tmp/
cp: cannot stat `Kost_me1.jpg': No such file or directory

The charset is correct, some folders use the right chars (üäö...) which
are displayed on server and client right.

I tested to access the share with windows client and can open the
picture, so i am able to access the files from windows.

Mounted are the shares with:
//web1.domain.tld/root$ /backup/src/web1   cifs
noauto,sec=ntlmv2,ver=1,cred=/root/.smb/web1.domain.tld 0 0

Samba on Server 3.6.3
Kernel on client 3.5.0-49-generic (Ubuntu)

Any Idea, how I get access to
1. the original file names (no translation to _)
2. can access the files (possible done, when 1. is done)

Thanks for you help,
Thomas

Re: character-problems

[Thomas Creutz]

Hi Jeff,

I installed/booted the mainline kernel for Ubuntu from
http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.14.2-utopic/ on the client.

But the problem is still the same (cant access the file). Also when I
can access the file, the other problem persists, that the name get
changed/translated.

Can this be changed on the samba server?

Thomas

Am 01.05.2014 11:35, schrieb Jeff Layton:
> On Thu, May 1, 2014 at 4:59 AM, Thomas Creutz <thomas.creutz-Mmb7MZpHnFY@public.gmane.org> wrote:
>> Hi List,
>>
>> I use samba server on my webserver and mount the share with a linux
>> client to backup the files remote. Problem is, that I cannot access all
>> files with linux cifs client and secondly filenames are changed, so that
>> the backup (when possible to access the files) is not well done.
>>
>> On the server output of ls -lah is:
>> insgesamt 400K
>> drwxr-xr-x  2 web178 client14 4,0K Mai 28  2008 .
>> drwxrwxrwx 13 web178 client14 4,0K Okt 17  2013 ..
>> -rw-r--r--  1 web178 client14  59K Mai 28  2008 JulianKiendlGitarre.jpg
>> -rw-r--r--  1 web178 client14  79K Mai 28  2008 Kost?me1.jpg
>> -rw-r--r--  1 web178 client14  51K Mai 28  2008 Kost?me2.jpg
>> -rw-r--r--  1 web178 client14  55K Mai 28  2008 MuellerReeb.jpg
>> -rw-r--r--  1 web178 client14  42K Mai 28  2008 Partyraum.jpg
>> -rw-r--r--  1 web178 client14  60K Mai 28  2008 Schlegel.jpg
>> -rw-r--r--  1 web178 client14  38K Mai 28  2008 Tanzen.jpg
>>
>> on the client side:
>> insgesamt 392K
>> drwxr-xr-x  2 1124 1051   0 Mai 28  2008 .
>> drwxrwxrwx 13 1124 1051   0 Okt 17  2013 ..
>> -rw-r--r--  1 1124 1051 59K Mai 28  2008 JulianKiendlGitarre.jpg
>> -rw-r--r--  1 1124 1051 79K Mai 28  2008 Kost_me1.jpg
>> -rw-r--r--  1 1124 1051 51K Mai 28  2008 Kost_me2.jpg
>> -rw-r--r--  1 1124 1051 55K Mai 28  2008 MuellerReeb.jpg
>> -rw-r--r--  1 1124 1051 42K Mai 28  2008 Partyraum.jpg
>> -rw-r--r--  1 1124 1051 60K Mai 28  2008 Schlegel.jpg
>> -rw-r--r--  1 1124 1051 38K Mai 28  2008 Tanzen.jpg
>>
>> LANG=C cp Kost_me1.jpg /tmp/
>> cp: cannot stat `Kost_me1.jpg': No such file or directory
>>
>> The charset is correct, some folders use the right chars (üäö...) which
>> are displayed on server and client right.
>>
>> I tested to access the share with windows client and can open the
>> picture, so i am able to access the files from windows.
>>
>> Mounted are the shares with:
>> //web1.domain.tld/root$ /backup/src/web1   cifs
>> noauto,sec=ntlmv2,ver=1,cred=/root/.smb/web1.domain.tld 0 0
>>
>> Samba on Server 3.6.3
>> Kernel on client 3.5.0-49-generic (Ubuntu)
>>
>> Any Idea, how I get access to
>> 1. the original file names (no translation to _)
>> 2. can access the files (possible done, when 1. is done)
>>
>> Thanks for you help,
>> Thomas
> 3.5 is pretty ancient by now, and the unicode handling in cifs got an
> overhaul a few months ago. Can you test with something more recent,
> like say, v3.14-ish kernel?
>
> -- Jeff

Re: character-problems

[Thomas Creutz]

Am 01.05.2014 12:54, schrieb Jeff Layton:
> Maybe...
>
> For your kernel, what is CONFIG_NLS_DEFAULT set to? I know that ubuntu
> used to set that to iso9959-1, but I think they recently moved to
> using utf8. If it's set to iso8559-1 then that may be related to the
> problem.

No, its not a charset problem.

grep CONFIG_NLS_DEFAULT /boot/config-$(uname -r)
CONFIG_NLS_DEFAULT="utf8"

because I also told, that the file names have the same problem on the
server (uploaded over php-scripts). Some scripts use utf-8 others
iso8...., thats why you will never see all chars right on console/samba
client.
But, this problem should get handelt by samba/cifs... the files are also
useable with a windows client (so the access problem is in the cifs
client, not the samba server)

Re: character-problems

[Thomas Creutz]

Am 01.05.2014 15:32, schrieb Jeff Layton:
> I'm not sure then.
>
> The client generally uses the same tables to convert between unicode
> and the local charset in both directions. One thing you could do is
> inspect some traces when you're reproducing this problem. Get a
> capture of both the "ls" and the "stat" (or cp) commands. Are the
> unicode names being sent by the server identical to the ones being
> sent by the client?

Ok, and how can I do this? tcpdump or some other debug ways?

Re: character-problems

[Günter Kukkukk]

Am 01.05.2014 13:11, schrieb Thomas Creutz:
> 
> Am 01.05.2014 12:54, schrieb Jeff Layton:
>> Maybe...
>>
>> For your kernel, what is CONFIG_NLS_DEFAULT set to? I know that ubuntu
>> used to set that to iso9959-1, but I think they recently moved to
>> using utf8. If it's set to iso8559-1 then that may be related to the
>> problem.
> 
> No, its not a charset problem.
> 
> grep CONFIG_NLS_DEFAULT /boot/config-$(uname -r)
> CONFIG_NLS_DEFAULT="utf8"
> 
> because I also told, that the file names have the same problem on the
> server (uploaded over php-scripts). Some scripts use utf-8 others
> iso8...., thats why you will never see all chars right on console/samba
> client.
> But, this problem should get handelt by samba/cifs... the files are also
> useable with a windows client (so the access problem is in the cifs
> client, not the samba server)

i've tested this on clients running
   opensuse-12.3 - kernel 3.7.10-1.28-desktop
   opensuse-13.1 - kernel 3.11.10-7-desktop
samba server Version 3.6.12-59.19.1-3208-SUSE-SL12.3-x86_64
is running on opensuse-12.3.

Here i don't see that
    Kost?me1.jpg  (i really used a question mark here)
on the server is changed to
    Kost_me1.jpg
on the client.

But in fact i really can't test that sanely here, because what is really
byte coded for "?" in Kost?me1.jpg on the server?
Can you get a hex-representation of that file name?

In addition it could also help when you post your samba server's smb.conf.

Cheers, Günter

--