Re: LInux NFSv4.1 client and server- case insensitive filesystems supported?

Re: LInux NFSv4.1 client and server- case insensitive filesystems supported?

[Cedric Blancher]

On Wed, 4 Jun 2025 at 19:58, Cedric Blancher <cedric.blancher@gmail.com> wrote:
>
> Good evening!
>
> Does the Linux NFSv4.1 client and server support case insensitive
> filesystems, e.g. exported FAT or NTFS?

Just found this in Linux kernel fs/nfsd/nfs4xdr.

        if (bmval0 & FATTR4_WORD0_CASE_INSENSITIVE) {
               p = xdr_reserve_space(xdr, 4);
               if (!p)
                       goto out_resource;
               *p++ = cpu_to_be32(0);
       }
       if (bmval0 & FATTR4_WORD0_CASE_PRESERVING) {
               p = xdr_reserve_space(xdr, 4);
               if (!p)
                       goto out_resource;
               *p++ = cpu_to_be32(1);
       }

How did this pass code review, ever?

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

Re: LInux NFSv4.1 client and server- case insensitive filesystems supported?

[Chuck Lever]

On 6/4/25 2:52 PM, Cedric Blancher wrote:
> On Wed, 4 Jun 2025 at 19:58, Cedric Blancher <cedric.blancher@gmail.com> wrote:
>>
>> Good evening!
>>
>> Does the Linux NFSv4.1 client and server support case insensitive
>> filesystems, e.g. exported FAT or NTFS?
> 
> Just found this in Linux kernel fs/nfsd/nfs4xdr.
> 
>         if (bmval0 & FATTR4_WORD0_CASE_INSENSITIVE) {
>                p = xdr_reserve_space(xdr, 4);
>                if (!p)
>                        goto out_resource;
>                *p++ = cpu_to_be32(0);
>        }
>        if (bmval0 & FATTR4_WORD0_CASE_PRESERVING) {
>                p = xdr_reserve_space(xdr, 4);
>                if (!p)
>                        goto out_resource;
>                *p++ = cpu_to_be32(1);
>        }
> 
> How did this pass code review, ever?

Hi Cedric-

I presume what you are complaining about is that when these attributes
are queried, NFSD returns fixed values rather than interrogating the
file system on which the target object resides.

Until very recently, the Linux dentry cache supported only case-
sensitive file name lookups, and all of the file systems that NFSD is
regularly tested with are case-preserving.

In that light, this code is entirely justified, and reflects very
similar existing behavior for NFSD's implementation of NFSv3 PATHCONF
(see nfsd3_proc_pathconf). Historically, on Linux, there is only one
possible correct answer for these settings.

My impression is that real case-insensitivity has been added to the
dentry cache in support of FAT on Android devices (or something like
that). That clears the path a bit for NFSD, but it needs to be
researched to see if that new support is adequate for NFS to use.

Recently, Roland poked me about NFSD support for case insensitivity.
Unfortunately that is not a simple topic. There are probably one or two
people on the mailing lists Cc'd here who can explore this with you so
we can understand exactly what behavior folks are looking for, and
determine whether it is feasible to support it in NFSD. Please keep the
discussion public because I'm sure there are multiple interested
parties and even more opinions.


-- 
Chuck Lever

Re: LInux NFSv4.1 client and server- case insensitive filesystems supported?

[Theodore Ts'o]

On Sat, Jun 07, 2025 at 02:30:37PM -0400, Chuck Lever wrote:
> 
> My impression is that real case-insensitivity has been added to the
> dentry cache in support of FAT on Android devices (or something like
> that). That clears the path a bit for NFSD, but it needs to be
> researched to see if that new support is adequate for NFS to use.

Case insensitivty was added in Linux in 2019, with the primary coding
work being done by Gabriel Krisman Bertazi of Collabora, and design
work being done being done by Gabriel, Michael Halcrow, and myself.
(Michael Halcrow in particular was responsible for devising how to
make case-insensitivity work with filename encryption and indexed
directories.)

The initial file systems that had case-insensitivty implemented was
ext4 and f2fs.  The initial use cases was Android devices (which had
used this horible wrapfs stacking file system thing which was trivial
to deadlock under stress, and its original reason for existing was
bug-for-bug compatibility with FAT), and for Steam so that Windows
games could have their expected case insensitivity.  (Collabora's work
was underwritten by Steam.)

There is an interesting write-up about NFS and case-insensitivity in a
relatively recent Internet-Draft[1], dated 2025-May-16.  In this I-D,
it points out that one of the primary concerns is that if the client
caches negative lookups under one case (say, MaDNeSS), and then the
file is created using a different case (say "madness"), then the
negative dentry cache indicating that MaDNeSS does not exist needs to
be removed when "madness" is created.  I'm not sure how Linux's NFS
client handles negative dentries, since even without
case-insensitivity, a file name that previously didn't exist could
have subsequently been created by another client on a different host.
So does Linux's NFS client simply does not use negative dentries, or
does it have some kind of cache invalidation scheme when the directory
has a new mtime, or some such?

[1] https://www.ietf.org/id/draft-ietf-nfsv4-internationalization-12.html#name-handling-of-string-equivale

Anyway, case sensitivity is one of those "interesting" problems which
has caused many headaches, including a potential security issue, and a
botched attempt to fix that security issue interacting poorly with
some of the more subtle design requirements so that file systems can
use tree-indexed directory lookups, even with case-insensitivty file
names and encrypted directory entries.  So in general, unless you have
strong financial backing where someone is willing to pay $$$ to
address a business-critical use case, my personal advice is to stay
far, far, away.  And I say this as a someone (with apologies to Linus)
who was partially responsible for Linux having case insensitivty
lookups in the first place.  :-)

Cheers,

						- Ted

Re: LInux NFSv4.1 client and server- case insensitive filesystems supported?

[Jeff Layton]

On Sat, 2025-06-07 at 22:39 +0000, Theodore Ts'o wrote:
> On Sat, Jun 07, 2025 at 02:30:37PM -0400, Chuck Lever wrote:
> > 
> > My impression is that real case-insensitivity has been added to the
> > dentry cache in support of FAT on Android devices (or something like
> > that). That clears the path a bit for NFSD, but it needs to be
> > researched to see if that new support is adequate for NFS to use.
> 
> Case insensitivty was added in Linux in 2019, with the primary coding
> work being done by Gabriel Krisman Bertazi of Collabora, and design
> work being done being done by Gabriel, Michael Halcrow, and myself.
> (Michael Halcrow in particular was responsible for devising how to
> make case-insensitivity work with filename encryption and indexed
> directories.)
> 
> The initial file systems that had case-insensitivty implemented was
> ext4 and f2fs.  The initial use cases was Android devices (which had
> used this horible wrapfs stacking file system thing which was trivial
> to deadlock under stress, and its original reason for existing was
> bug-for-bug compatibility with FAT), and for Steam so that Windows
> games could have their expected case insensitivity.  (Collabora's work
> was underwritten by Steam.)
> 
> There is an interesting write-up about NFS and case-insensitivity in a
> relatively recent Internet-Draft[1], dated 2025-May-16.  In this I-D,
> it points out that one of the primary concerns is that if the client
> caches negative lookups under one case (say, MaDNeSS), and then the
> file is created using a different case (say "madness"), then the
> negative dentry cache indicating that MaDNeSS does not exist needs to
> be removed when "madness" is created.  I'm not sure how Linux's NFS
> client handles negative dentries, since even without
> case-insensitivity, a file name that previously didn't exist could
> have subsequently been created by another client on a different host.
> So does Linux's NFS client simply does not use negative dentries, or
> does it have some kind of cache invalidation scheme when the directory
> has a new mtime, or some such?
> 

It's a little more complicated than that, but that's basically right:

When the attrs on the parent directory indicate that there has been a
change to it since a dentry was last revalidated, the client will
return <=0 to d_revalidate attempts on the dentry. That will make the
VFS re-drive a lookup instead of trusting the cache.


> [1] https://www.ietf.org/id/draft-ietf-nfsv4-internationalization-12.html#name-handling-of-string-equivale
> 
> Anyway, case sensitivity is one of those "interesting" problems which
> has caused many headaches, including a potential security issue, and a
> botched attempt to fix that security issue interacting poorly with
> some of the more subtle design requirements so that file systems can
> use tree-indexed directory lookups, even with case-insensitivty file
> names and encrypted directory entries.  So in general, unless you have
> strong financial backing where someone is willing to pay $$$ to
> address a business-critical use case, my personal advice is to stay
> far, far, away.  And I say this as a someone (with apologies to Linus)
> who was partially responsible for Linux having case insensitivty
> lookups in the first place.  :-)

-- 
Jeff Layton <jlayton@kernel.org>

Re: LInux NFSv4.1 client and server- case insensitive filesystems supported?

[Chuck Lever]

On 6/7/25 6:39 PM, Theodore Ts'o wrote:
> On Sat, Jun 07, 2025 at 02:30:37PM -0400, Chuck Lever wrote:
>>
>> My impression is that real case-insensitivity has been added to the
>> dentry cache in support of FAT on Android devices (or something like
>> that). That clears the path a bit for NFSD, but it needs to be
>> researched to see if that new support is adequate for NFS to use.
> 
> Case insensitivty was added in Linux in 2019, with the primary coding
> work being done by Gabriel Krisman Bertazi of Collabora, and design
> work being done being done by Gabriel, Michael Halcrow, and myself.
> (Michael Halcrow in particular was responsible for devising how to
> make case-insensitivity work with filename encryption and indexed
> directories.)

For some reason I thought case-insensitivity support was merged more
recently than that. I recall it first appearing as a session at LSF in
Park City, but maybe that one was in 2018.


> There is an interesting write-up about NFS and case-insensitivity in a
> relatively recent Internet-Draft[1], dated 2025-May-16.  In this I-D,
> it points out that one of the primary concerns is that if the client
> caches negative lookups under one case (say, MaDNeSS), and then the
> file is created using a different case (say "madness"), then the
> negative dentry cache indicating that MaDNeSS does not exist needs to
> be removed when "madness" is created.  I'm not sure how Linux's NFS
> client handles negative dentries, since even without
> case-insensitivity, a file name that previously didn't exist could
> have subsequently been created by another client on a different host.
> So does Linux's NFS client simply does not use negative dentries, or
> does it have some kind of cache invalidation scheme when the directory
> has a new mtime, or some such?
> 
> [1] https://www.ietf.org/id/draft-ietf-nfsv4-internationalization-12.html#name-handling-of-string-equivale

nfs(5) describes the lookupcache= mount option. It controls how the
Linux NFS client caches positive and negative lookup results.


-- 
Chuck Lever

Re: LInux NFSv4.1 client and server- case insensitive filesystems supported?

[Theodore Ts'o]

On Sun, Jun 08, 2025 at 12:29:30PM -0400, Chuck Lever wrote:
> 
> For some reason I thought case-insensitivity support was merged more
> recently than that. I recall it first appearing as a session at LSF in
> Park City, but maybe that one was in 2018.

commit b886ee3e778ec2ad43e276fd378ab492cf6819b7
Author: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
Date:   Thu Apr 25 14:12:08 2019 -0400

    ext4: Support case-insensitive file name lookups

> nfs(5) describes the lookupcache= mount option. It controls how the
> Linux NFS client caches positive and negative lookup results.

Has anyone just tried it?  It might just work.  To create a
case-folded directory:

# mke2fs -Fq -t ext4 -O casefold /dev/vdc
# mount /dev/vdc /vdc
# mkdir /vdc/casefold
# chattr +F /vdc/casefold
# cp /etc/issue /vdc/casefold/MaDNeSS
# cat /vdc/casefold/madness

Then export the directory and mount it via NFS, and let us know how it
goes.  I'm currently on a cruise ship so it's a bit harder for me to
do the experiment myself.  :-)

						- Ted

Re: LInux NFSv4.1 client and server- case insensitive filesystems supported?

[Chuck Lever]

On 6/8/25 4:52 PM, Theodore Ts'o wrote:
> On Sun, Jun 08, 2025 at 12:29:30PM -0400, Chuck Lever wrote:
>>
>> For some reason I thought case-insensitivity support was merged more
>> recently than that. I recall it first appearing as a session at LSF in
>> Park City, but maybe that one was in 2018.
> 
> commit b886ee3e778ec2ad43e276fd378ab492cf6819b7
> Author: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
> Date:   Thu Apr 25 14:12:08 2019 -0400
> 
>     ext4: Support case-insensitive file name lookups
> 
>> nfs(5) describes the lookupcache= mount option. It controls how the
>> Linux NFS client caches positive and negative lookup results.
> 
> Has anyone just tried it?  It might just work.  To create a
> case-folded directory:
> 
> # mke2fs -Fq -t ext4 -O casefold /dev/vdc
> # mount /dev/vdc /vdc
> # mkdir /vdc/casefold
> # chattr +F /vdc/casefold
> # cp /etc/issue /vdc/casefold/MaDNeSS
> # cat /vdc/casefold/madness
> 
> Then export the directory and mount it via NFS, and let us know how it
> goes.  I'm currently on a cruise ship so it's a bit harder for me to
> do the experiment myself.  :-)

NFSD currently asserts that all exported file systems are case-sensitive
and case-preserving (either via NFSv3 or NFSv4). There is very likely
some additional work necessary.

Ted, do you happen to know if there are any fstests that exercise case-
insensitive lookups? I would not regard that simple test as "job done!
put pencil down!" :-)


-- 
Chuck Lever

Re: LInux NFSv4.1 client and server- case insensitive filesystems supported?

[Gabriel Krisman Bertazi]

Chuck Lever <chuck.lever@oracle.com> writes:

> On 6/8/25 4:52 PM, Theodore Ts'o wrote:
>> On Sun, Jun 08, 2025 at 12:29:30PM -0400, Chuck Lever wrote:
>>>
>>> For some reason I thought case-insensitivity support was merged more
>>> recently than that. I recall it first appearing as a session at LSF in
>>> Park City, but maybe that one was in 2018.

Hi Chuck,

The first LSF discussion on this implementation was Park City, 2018. It
was merged early 2019.

> Ted, do you happen to know if there are any fstests that exercise case-
> insensitive lookups? I would not regard that simple test as "job done!
> put pencil down!" :-)

generic/556 tests basic semantics and many corner cases of casefolded
lookups.

-- 
Gabriel Krisman Bertazi

Re: LInux NFSv4.1 client and server- case insensitive filesystems supported?

[Theodore Ts'o]

On Sun, Jun 08, 2025 at 05:52:36PM -0400, Chuck Lever wrote:
> NFSD currently asserts that all exported file systems are case-sensitive
> and case-preserving (either via NFSv3 or NFSv4). There is very likely
> some additional work necessary.

If the underlying file system on the server side does do case
insensitive lookups, how badly would it confuse the NFS client if a
lookup of "MaDNeSS" and "maddness" both worked and returned the same
file, even though readdir(2) only showed the existence of "MaDNeSS"
--- despite the fact that the nfsd asserted that file system was
case-sensitive?

> Ted, do you happen to know if there are any fstests that exercise case-
> insensitive lookups? I would not regard that simple test as "job done!
> put pencil down!" :-)

There are.   See:

common/casefold
tests/f2fs/012
tests/generic/453
tests/generic/454
tests/generic/556

You'll need to make some adjustments in common/casefold for NFS,
though.  The tests also assume that case insensivity can be adjusted
on a per-directory basis, using chattr +F and chattr -F, and that
probably isn't going to port over to NFS, so you might need to adjust
the tests as well.

Note that some of these tests also are checking Unicode case-folding
rules, which is a bit different from the ASCII case-folding which FAT
implemented.  It also might be interesting/amsuing to see what happens
if you ran these tests where the NFS server was exporting, say, a
case-folded file system from a MacOS server, or a Windows NFS server,
and the client was running Linux NFS.  Or what might happen if you
tried to do the same thing using, say, CIFS.   :-)

Cheers,

       	       	    	 	     	- Ted

Re: LInux NFSv4.1 client and server- case insensitive filesystems supported?

[Chuck Lever]

On 6/9/25 11:50 AM, Theodore Ts'o wrote:
> On Sun, Jun 08, 2025 at 05:52:36PM -0400, Chuck Lever wrote:
>> NFSD currently asserts that all exported file systems are case-sensitive
>> and case-preserving (either via NFSv3 or NFSv4). There is very likely
>> some additional work necessary.
> 
> If the underlying file system on the server side does do case
> insensitive lookups, how badly would it confuse the NFS client if a
> lookup of "MaDNeSS" and "maddness" both worked and returned the same
> file, even though readdir(2) only showed the existence of "MaDNeSS"
> --- despite the fact that the nfsd asserted that file system was
> case-sensitive?

IIRC the Linux NFS client used to cache READDIR results via the dcache.
It doesn't do that, these days. If the client mounted with
lookupcache=none, it might work OK?


>> Ted, do you happen to know if there are any fstests that exercise case-
>> insensitive lookups? I would not regard that simple test as "job done!
>> put pencil down!" :-)
> 
> There are.   See:
> 
> common/casefold
> tests/f2fs/012
> tests/generic/453
> tests/generic/454
> tests/generic/556
> 
> You'll need to make some adjustments in common/casefold for NFS,
> though.  The tests also assume that case insensivity can be adjusted
> on a per-directory basis, using chattr +F and chattr -F, and that
> probably isn't going to port over to NFS, so you might need to adjust
> the tests as well.

This is probably the more general concern:

- Both the NFSv3 and NFSv4 protocols mark a whole file system as either
  case sensitive or case insensitive.

- NFS protocols do not facilitate case-sensitivity to be be enabled or
  disabled from NFS clients.

It sounds like it would be difficult for NFS clients to make sense of
an exported extN file system that contained some case sensitive and some
case insensitive directories.


> Note that some of these tests also are checking Unicode case-folding
> rules, which is a bit different from the ASCII case-folding which FAT
> implemented.  It also might be interesting/amsuing to see what happens
> if you ran these tests where the NFS server was exporting, say, a
> case-folded file system from a MacOS server, or a Windows NFS server,
> and the client was running Linux NFS.  Or what might happen if you
> tried to do the same thing using, say, CIFS.   :-)

Re-exporting remote file systems is a massive jungle even on good days.

I think a narrower concern might be keeping lookup behavior consistent
between local accessors, NFS accessors, and accessors via SMB (Samba or
ksmbd). I admit I have no idea if that's possible.


-- 
Chuck Lever

Re: LInux NFSv4.1 client and server- case insensitive filesystems supported?

[Christoph Hellwig]

On Sat, Jun 07, 2025 at 02:30:37PM -0400, Chuck Lever wrote:
> Until very recently, the Linux dentry cache supported only case-
> sensitive file name lookups, and all of the file systems that NFSD is
> regularly tested with are case-preserving.

Linux has supported case insensitive file system since 1992 when Werner
added the original msdos FAT support, i.e. it exists much longer than
the dcache or knfsd.

Specific support for dealing with case insensitive in the dcache instead
working around it was added in 2008 for the case insensitive XFS
directories in 2008:

commit 9403540c0653122ca34884a180439ddbfcbcb524
Author: Barry Naujok <bnaujok@sgi.com>
Date:   Wed May 21 16:50:46 2008 +1000

    dcache: Add case-insensitive support d_ci_add() routine

That being said no one ever intended any of these to be exported over
NFS, and I also question the sanity of anyone wanting to use case
insensitive file systems over NFS.

Re: LInux NFSv4.1 client and server- case insensitive filesystems supported?

[Chuck Lever]

On 6/9/25 1:57 AM, Christoph Hellwig wrote:
> On Sat, Jun 07, 2025 at 02:30:37PM -0400, Chuck Lever wrote:
>> Until very recently, the Linux dentry cache supported only case-
>> sensitive file name lookups, and all of the file systems that NFSD is
>> regularly tested with are case-preserving.
> 
> Linux has supported case insensitive file system since 1992 when Werner
> added the original msdos FAT support, i.e. it exists much longer than
> the dcache or knfsd.
> 
> Specific support for dealing with case insensitive in the dcache instead
> working around it was added in 2008 for the case insensitive XFS
> directories in 2008:
> 
> commit 9403540c0653122ca34884a180439ddbfcbcb524
> Author: Barry Naujok <bnaujok@sgi.com>
> Date:   Wed May 21 16:50:46 2008 +1000
> 
>     dcache: Add case-insensitive support d_ci_add() routine

My memory must be quite faulty then. I remember there being significant
controversy at the Park City LSF around some patches adding support for
case insensitivity. But so be it -- I must not have paid terribly close
attention due to lack of oxygen.


> That being said no one ever intended any of these to be exported over
> NFS, and I also question the sanity of anyone wanting to use case
> insensitive file systems over NFS.

My sense is that case insensitivity for NFS exports is for Windows-based
clients and/or compatibility with Samba / SMB clients. But it does open
up a whole bunch of twisty little corner cases that I'm not terribly
anxious to bite off and chew on (See the i18n Internet Draft that Ted
cited earlier just as a start).

Perhaps if we can narrow down the requirements and deployment
environments, some limited form of case-insensitivity support for NFS
might start to make sense.

Does it, for example, make sense for NFSD to query the file system
on its case sensitivity when it prepares an NFSv3 PATHCONF response?
Or perhaps only for NFSv4, since NFSv4 pretends to have some recognition
of internationalized file names?


-- 
Chuck Lever

Re: LInux NFSv4.1 client and server- case insensitive filesystems supported?

[Christoph Hellwig]

On Mon, Jun 09, 2025 at 10:16:24AM -0400, Chuck Lever wrote:
> > Date:   Wed May 21 16:50:46 2008 +1000
> > 
> >     dcache: Add case-insensitive support d_ci_add() routine
> 
> My memory must be quite faulty then. I remember there being significant
> controversy at the Park City LSF around some patches adding support for
> case insensitivity. But so be it -- I must not have paid terribly close
> attention due to lack of oxygen.

Well, that is when the ext4 CI code landed, which added the unicode
normalization, and with that another whole bunch of issues.

> > That being said no one ever intended any of these to be exported over
> > NFS, and I also question the sanity of anyone wanting to use case
> > insensitive file systems over NFS.
> 
> My sense is that case insensitivity for NFS exports is for Windows-based
> clients

I still question the sanity of anyone using a Windows NFS client in
general, but even more so on a case insensitive file system :)


> Does it, for example, make sense for NFSD to query the file system
> on its case sensitivity when it prepares an NFSv3 PATHCONF response?
> Or perhaps only for NFSv4, since NFSv4 pretends to have some recognition
> of internationalized file names?

Linus hates pathconf any anything like it with passion.  Altough we
basically got it now with statx by tacking it onto a fast path
interface instead, which he now obviously also hates.  But yes, nfsd
not beeing able to query lots of attributes, including actual important
ones is largely due to the lack of proper VFS interfaces.