Support for filesystems with d_revalidate (NFS)

Support for filesystems with d_revalidate (NFS)

[Caldwell, Blake A.]

Hi there,

Is perhaps a fix for disabling overlayfsroot for filesystems with revalidate on the horizon? My use case for overlayfs is a NFS (v3 or v4) lowerdir, which appears rejected by the code below because NFS defines d_revalidate, d_automount, and d_weak_revalidate (v3 only).

There has been discussion in this launchpad bug about xattr copyup problems:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1039402

fs/overlayfs/super.c@692:
       /*
        * We don't support:
        *  - automount filesystems
        *  - filesystems with revalidate (FIXME for lower layer)
        *  - filesystems with case insensitive names
        */
       if (dop &&
           (dop->d_manage || dop->d_automount ||
            dop->d_revalidate || dop->d_weak_revalidate ||
            dop->d_compare || dop->d_hash)) {
               return false;
       }

Regarding the referenced code in overlayfs-next, is there a condition in which NFSv4 could be supported, such as if mounted read-only?

-Blake

Re: Support for filesystems with d_revalidate (NFS)

[Eric Jones]

I don't have an answer, but I'll note it worked at some point. For example, the overlayfs code in SuSE's SLES12 release allows it. I'm not sure why Novell merged such a different version of the overlayfs code.


On Mon, Feb 09, 2015 at 03:11:46PM +0000, Caldwell, Blake A. wrote:
> Hi there,
> 
> Is perhaps a fix for disabling overlayfsroot for filesystems with revalidate on the horizon? My use case for overlayfs is a NFS (v3 or v4) lowerdir, which appears rejected by the code below because NFS defines d_revalidate, d_automount, and d_weak_revalidate (v3 only).
> 
> There has been discussion in this launchpad bug about xattr copyup problems:
> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1039402
> 
> fs/overlayfs/super.c@692:
>        /*
>         * We don't support:
>         *  - automount filesystems
>         *  - filesystems with revalidate (FIXME for lower layer)
>         *  - filesystems with case insensitive names
>         */
>        if (dop &&
>            (dop->d_manage || dop->d_automount ||
>             dop->d_revalidate || dop->d_weak_revalidate ||
>             dop->d_compare || dop->d_hash)) {
>                return false;
>        }
> 
> Regarding the referenced code in overlayfs-next, is there a condition in which NFSv4 could be supported, such as if mounted read-only?
> 
> -Blake
> --
> To unsubscribe from this list: send the line "unsubscribe linux-unionfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Support for filesystems with d_revalidate (NFS)

[Caldwell, Blake A.]

Thanks. That could help in the short-term. I'd hope to see see this in mainline eventually. If someone could elaborate on the blanket exclusion of d_revalidate, I'd like to see if the task to add read-only NFS support is reasonable.

-Blake


On Feb 9, 2015, at 2:12 PM, Eric Jones <ejones@cray.com> wrote:

> I don't have an answer, but I'll note it worked at some point. For example, the overlayfs code in SuSE's SLES12 release allows it. I'm not sure why Novell merged such a different version of the overlayfs code.
> 
> 
> On Mon, Feb 09, 2015 at 03:11:46PM +0000, Caldwell, Blake A. wrote:
>> Hi there,
>> 
>> Is perhaps a fix for disabling overlayfsroot for filesystems with revalidate on the horizon? My use case for overlayfs is a NFS (v3 or v4) lowerdir, which appears rejected by the code below because NFS defines d_revalidate, d_automount, and d_weak_revalidate (v3 only).
>> 
>> There has been discussion in this launchpad bug about xattr copyup problems:
>> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1039402
>> 
>> fs/overlayfs/super.c@692:
>>       /*
>>        * We don't support:
>>        *  - automount filesystems
>>        *  - filesystems with revalidate (FIXME for lower layer)
>>        *  - filesystems with case insensitive names
>>        */
>>       if (dop &&
>>           (dop->d_manage || dop->d_automount ||
>>            dop->d_revalidate || dop->d_weak_revalidate ||
>>            dop->d_compare || dop->d_hash)) {
>>               return false;
>>       }
>> 
>> Regarding the referenced code in overlayfs-next, is there a condition in which NFSv4 could be supported, such as if mounted read-only?
>> 
>> -Blake
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-unionfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Support for filesystems with d_revalidate (NFS)

[Miklos Szeredi]

On Tue, Feb 10, 2015 at 7:26 PM, Caldwell, Blake A. <blakec@ornl.gov> wrote:
> Thanks. That could help in the short-term. I'd hope to see see this in mainline eventually. If someone could elaborate on the blanket exclusion of d_revalidate, I'd like to see if the task to add read-only NFS support is reasonable.

It was easier to merge upstream that way (the acutal change was
suggested by Al Viro).  We could add it back if we can show that it
doesn't hurt.

Generally distributed fs need ->d_revalidate because filesystem can
change without any write activity on the local mount (and even if
mounted read-only) to check for remote changes.

Overlayfs assumes that lower fs is constant and behavior is undefined
if the underlying filesystem changes.  So basically we don't care
about d_revalidate, but may still want to call it due to possible
assumptions about it being called by the VFS.

So easy option is to call ->d_revalidate and error out if it returns "invalid"

Thanks,
Miklos

Re: Support for filesystems with d_revalidate (NFS)

[David Howells]

Caldwell, Blake A. <blakec@ornl.gov> wrote:

> My use case for overlayfs is a NFS (v3 or v4) lowerdir, which appears
> rejected by the code below because NFS defines d_revalidate, d_automount,
> and d_weak_revalidate (v3 only).

How do you solve the file locking and fanotify problems?

David

Re: Support for filesystems with d_revalidate (NFS)

[Caldwell, Blake A.]

On Feb 11, 2015, at 6:03 AM, David Howells <dhowells@redhat.com<mailto:dhowells@redhat.com>> wrote:

Caldwell, Blake A. <blakec@ornl.gov<mailto:blakec@ornl.gov>> wrote:

My use case for overlayfs is a NFS (v3 or v4) lowerdir, which appears
rejected by the code below because NFS defines d_revalidate, d_automount,
and d_weak_revalidate (v3 only).

How do you solve the file locking and fanotify problems?
That's where I need your help :-)

You're lkml post was helpful for me to read
https://lkml.org/lkml/2015/1/26/740

NFS is a bit overkill for distributing a filesystem tree snapshot over the network, but it is an efficient way to do it. Pages are loaded into the page cache, meaning the whole image doesn't need to be transferred over the network. Coherent locking and *notify are complicating the matter, but I could see how supporting them is necessary to declare NFS support. I'm not sure what options exist for this use case, but it seems to be shared by several others.

To describe the desired behavior. When node1 opens a file R/O a local read lock is sufficient. It would prevent processes local to the system from opening an overlapping write lock. If another node, node2, opens the same file R/W it should attempt to take a lock that would conflict with the lock on node1. However, it will invoke copyup on node2 and maintain a write lock. Unintended copyups need to be blocked, or (as you mentioned without proper handling) the resulting copy may lose the lock.

This seems to point towards local only locks -o nlock, and trusting the cluster not to modify the filesystem (to also prevent fanotify). Given the lack of enforcement mechanisms on the client for latter, are we limited to removing that check by hand and compiling a custom kernel, or would an override "mount with possible inconsistency" option be merge-able?