vfat file system does not allow a space at the end of a directory name

vfat file system does not allow a space at the end of a directory name

[Chris Ward]

I had a zip file (from downloading pictures from Google Photos) where
some of the directory names had a space at the end. This unzipped fine
on an ext4 file system, but gave errors when unpacking onto a USB key
which had a FAT file system.

On further investigation, file names have the same problem. On my USB key, I get

tjcw@tjcw-OptiPlex-7010:/media/tjcw/Photos$ touch 'aab '
touch: setting times of 'aab ': No such file or directory
tjcw@tjcw-OptiPlex-7010:/media/tjcw/Photos$ echo ok >'aab '
bash: aab : Invalid argument
tjcw@tjcw-OptiPlex-7010:/media/tjcw/Photos$ touch 'aab'
tjcw@tjcw-OptiPlex-7010:/media/tjcw/Photos$ echo ok >'aab'
tjcw@tjcw-OptiPlex-7010:/media/tjcw/Photos$

Is this as expected, or is it a bug in the kernel's handling of FAT
file systems ? Probably the next step is to check the behaviour under
Microsoft Windows.

Chris Ward

Re: vfat file system does not allow a space at the end of a directory name

[Thomas Weißschuh]

Hi Chris,

On 2023-07-30 23:33:03+0100, Chris Ward wrote:
> I had a zip file (from downloading pictures from Google Photos) where
> some of the directory names had a space at the end. This unzipped fine
> on an ext4 file system, but gave errors when unpacking onto a USB key
> which had a FAT file system.
> 
> On further investigation, file names have the same problem. On my USB key, I get
> 
> tjcw@tjcw-OptiPlex-7010:/media/tjcw/Photos$ touch 'aab '
> touch: setting times of 'aab ': No such file or directory
> tjcw@tjcw-OptiPlex-7010:/media/tjcw/Photos$ echo ok >'aab '
> bash: aab : Invalid argument
> tjcw@tjcw-OptiPlex-7010:/media/tjcw/Photos$ touch 'aab'
> tjcw@tjcw-OptiPlex-7010:/media/tjcw/Photos$ echo ok >'aab'
> tjcw@tjcw-OptiPlex-7010:/media/tjcw/Photos$
> 
> Is this as expected, or is it a bug in the kernel's handling of FAT
> file systems ? Probably the next step is to check the behaviour under
> Microsoft Windows.

AFAIK this is a limitation in the FAT filesystem itself.

The problem is that FAT stores filenames in fixed width fields.
Unused space in these fields is filled with ' '.
This means that storing "foo", "foo " or "foo ". would all result in the
exact same result and it will be impossible to figure out the correct
one afterwards.

From Wikipedia [0]:

    trailing spaces in either the base name or the extension are
    considered to be padding and not a part of the file name

From fs/fat/namei_vfat.c vfat_is_used_badchars():

    last character cannot be space

[0] https://en.wikipedia.org/wiki/Design_of_the_FAT_file_system

Re: vfat file system does not allow a space at the end of a directory name

[Theodore Ts'o]

On Mon, Jul 31, 2023 at 07:55:39AM +0200, Thomas Weißschuh wrote:
> On 2023-07-30 23:33:03+0100, Chris Ward wrote:
> > I had a zip file (from downloading pictures from Google Photos) where
> > some of the directory names had a space at the end. This unzipped fine
> > on an ext4 file system, but gave errors when unpacking onto a USB key
> > which had a FAT file system.
>
> AFAIK this is a limitation in the FAT filesystem itself.
> 
> The problem is that FAT stores filenames in fixed width fields.
> Unused space in these fields is filled with ' '.
> This means that storing "foo", "foo " or "foo ". would all result in the
> exact same result and it will be impossible to figure out the correct
> one afterwards.

Yeah, I'd argue that this is a misfeature in Google Photos; its
product manager should strongly consider stripping trailing spaces
from picture titles.

Disclosure: I work for Google but not anywhere near the Google Photos
product team, and this is my own opinion not Google's.  Feel free to
file product feedback in the Google Photos app, though.  :-)

						- Ted