exFAT unexpected handling of filenames ending with dots

exFAT unexpected handling of filenames ending with dots

[Aidan MacDonald]

Hi, Namjae and Sungjong,

Recently, I was made aware of a problem with how the exFAT driver handles filenames ending with dots.

Original bug report was against an audio player supported by Rockbox:
https://www.rockbox.org/tracker/task/13293

Upon further investigation it turned out to be a Linux kernel issue. Note the audio player referenced there runs Linux 3.10 or so and uses some version of the Samsung exFAT driver -- so I guess this has been an issue for a _long_ time. I was able to reproduce it on my laptop running v5.10.39!

It appears that any number of trailing dots are stripped from the end of the filename, causing some interesting bugs.

The behaviour I am observing is this:

1. If creating a file, the name is stripped of all trailing dots and the stripped name is used to create the file (original name is silently discarded).

2. If accessing a file within a directory, the stripped filename is used to conduct the search, ie. if you enter 'A...' the driver will actually search using the name 'A'.

It is this second part which causes problems. If you have a file named "A." on an exFAT filesystem, it will show up in directory listings but if you try to access it, you get 'file not found'. That is because the driver is actually looking for "A" even though you think you are looking for "A." -- and even worse, if "A" does exist, the driver will silently access "A" instead!

Clearly due to the first part, you cannot get into this situation without using another driver -- like the exFAT FUSE driver -- to create the problematic filenames. (That's how the Rockbox bug reporter managed to run into this.)

Now, a function called exfat_striptail_len() in fs/exfat/namei.c is responsible for the filename stripping, it simply removes all the trailing dots from a name and I guess it is the cause of this problem. So this 'feature' was intentionally added in.

I've only skimmed the exFAT spec but I can find nothing in it about stripping dots from the end of a filename. The FUSE-based exFAT driver appears to treat dots as significant too.

It seems Windows suffers the same trailing dots bug, silently accessing the wrong files despite listing all names correctly. But I obviously can't say whether that is due to filesystem drivers or issues higher up the stack.

To be honest I have no idea what the purpose of this 'dot stripping' is... even if it was for the sake of "Windows compatibility" -- ie. mimicking Windows bugs -- there are names that Windows normally rejects which the in-kernel exFAT driver will accept, such as names with trailing spaces.

Personally, I don't see any issue with how the FUSE driver behaves. It also seems to be correct with respect to Microsoft's official spec. I don't see why Linux should deviate from the spec, especially in a way that makes it _less_ robust.

I did search for any other reports of this issue, but it seems to be such a corner case that nobody's mentioned it anywhere. Nor can I find any discussion or rationale for the dot stripping behaviour.

Kind regards,
Aidan

RE: exFAT unexpected handling of filenames ending with dots

[Namjae Jeon]

> Hi, Namjae and Sungjong,
Hi Aidan,
> 
> Recently, I was made aware of a problem with how the exFAT driver handles filenames ending with dots.
> 
> Original bug report was against an audio player supported by Rockbox:
> https://protect2.fireeye.com/v1/url?k=f989dd16-a612e473-f9885659-000babff32e3-
> 714879f4126cb358&q=1&e=493a850d-5944-4bc5-a1c0-
> 636611a9023d&u=https%3A%2F%2Fwww.rockbox.org%2Ftracker%2Ftask%2F13293
> 
> Upon further investigation it turned out to be a Linux kernel issue. Note the audio player referenced
> there runs Linux 3.10 or so and uses some version of the Samsung exFAT driver -- so I guess this has
> been an issue for a _long_ time. I was able to reproduce it on my laptop running v5.10.39!
> 
> It appears that any number of trailing dots are stripped from the end of the filename, causing some
> interesting bugs.
> 
> The behaviour I am observing is this:
> 
> 1. If creating a file, the name is stripped of all trailing dots and the stripped name is used to
> create the file (original name is silently discarded).
> 
> 2. If accessing a file within a directory, the stripped filename is used to conduct the search, ie. if
> you enter 'A...' the driver will actually search using the name 'A'.
> 
> It is this second part which causes problems. If you have a file named "A." on an exFAT filesystem, it
> will show up in directory listings but if you try to access it, you get 'file not found'. That is
> because the driver is actually looking for "A" even though you think you are looking for "A." -- and
> even worse, if "A" does exist, the driver will silently access "A" instead!
> 
> Clearly due to the first part, you cannot get into this situation without using another driver -- like
> the exFAT FUSE driver -- to create the problematic filenames. (That's how the Rockbox bug reporter
> managed to run into this.)
> 
> Now, a function called exfat_striptail_len() in fs/exfat/namei.c is responsible for the filename
> stripping, it simply removes all the trailing dots from a name and I guess it is the cause of this
> problem. So this 'feature' was intentionally added in.
> 
> I've only skimmed the exFAT spec but I can find nothing in it about stripping dots from the end of a
> filename. The FUSE-based exFAT driver appears to treat dots as significant too.
> 
> It seems Windows suffers the same trailing dots bug, silently accessing the wrong files despite
> listing all names correctly. But I obviously can't say whether that is due to filesystem drivers or
> issues higher up the stack.
We have seen the phrase on the exFAT specification below and checked Windows behavior, So we have
interpreted that exFAT also has same trailing period rule as described in the FAT specification.

#exFAT
The concatenated file name has the same set of illegal characters as other FAT-based file
systems (see Table 31).

#FAT
Long Directory Entries
Long names are limited to 255 characters, not including the trailing NUL. The total path length of a
long name cannot exceed 260 characters, including the trailing NUL. The characters may be any
combination of those defined for short names with the addition of the period (.) character used
multiple times within the long name. A space is also a valid character in a long name as it always has
been for a short name. However, in short names it typically is not used. The following six special
characters are now allowed in a long name. They are not legal in a short name.
+ , ; = [ ]
Embedded spaces within a long name are allowed. Leading and trailing spaces in a long name are
ignored.
Leading and embedded periods are allowed in a name and are stored in the long name. Trailing
periods are ignored.

> To be honest I have no idea what the purpose of this 'dot stripping' is... even if it was for the sake
> of "Windows compatibility" -- ie. mimicking Windows bugs -- there are names that Windows normally
> rejects which the in-kernel exFAT driver will accept, such as names with trailing spaces.
Let me check it. There was a code to strip names with trailing spaces before. But it was removed with some
reason. We need to check our history. If needed, We will add it again.
> 
> Personally, I don't see any issue with how the FUSE driver behaves. It also seems to be correct with
> respect to Microsoft's official spec. I don't see why Linux should deviate from the spec, especially
> in a way that makes it _less_ robust.
I think it is a problem of fuse-exfat, not Windows and kernel exfat. fuse-exfat allow to create file with
invalid name that is not recognized in Windows.
> 
> I did search for any other reports of this issue, but it seems to be such a corner case that nobody's
> mentioned it anywhere. Nor can I find any discussion or rationale for the dot stripping behaviour.
> 
> Kind regards,
> Aidan