Re: shiftfs status and future development

[Seth Forshee <seth.forshee@canonical.com>]

On Mon, Jun 18, 2018 at 08:11:52PM +0300, Amir Goldstein wrote:
> On Mon, Jun 18, 2018 at 5:56 PM, James Bottomley
> <James.Bottomley@hansenpartnership.com> wrote:
> [...]
> >> > >  - Does not break inotify
> >> >
> >> > I don't expect it does, but I haven't checked.
> >>
> >> I haven't checked either; I'm planning to do so soon. This is a
> >> concern that was expressed to me by others, I think because inotify
> >> doesn't work with overlayfs.
> >
> > I think shiftfs does work simply because it doesn't really do overlays,
> > so lots of stuff that doesn't work with overlays does work with it.
> >
> 
> I'm afraid shiftfs suffers from the same problems that the old naiive
> overlayfs inode implementation suffered from.
> 
> This problem is demonstrated with LTP tests inotify08 inotify09.
> shiftfs_new_inode() is called on every lookup, so inotify watch
> may be set on an inode object, then dentry is evicted from cache
> and then all events on new dentry are not reported on the watched
> inode. You will need to implement hashed inodes to solve it.
> Can be done as overlay does - hashing by real inode pointer.

Thanks for the pointer. I modified the LTP inotify08 test to use shiftfs
instead of overlayfs, and I can confirm that it fails to see the events.

> This is just one of those subtle things about stacked fs and there may
> be other in present and more in future - if we don't have a shared code
> base for the two stacked fs, I wager you are going to end up "cherry
> picking" fixes often.
> 
> IMO, an important question to ask is, since both shiftfs and overlayfs
> are strongly coupled with container use cases, are there users that
> are interested in both layering AND shifting? on the same "mark"?
> If the answer is yes, then this may be an argument in favor of
> integrating at least some of shittfs functionality into overlayfs.
> 
> Another argument is that shiftfs itself takes the maximum allowed
> 2 levels of s_stack_depth for it's 2 mounts, so it is actually not
> possible with current VFS limitation to combine shiftfs with overlayfs.

That's unfortunate -- it will prevent nested use in addition to use with
overlayfs.

I have been wondering if shiftfs could be made to detect when it was
being layered on top of a shiftfs mount, and do something to make it
behave like it was a single shift on top of the original subtree. I may
look into this. Of course it doesn't help with overlayfs.

> This could be solved relatively easily by adding "-o mark" support
> to overlayfs and allowing to mount shiftfs also over "marked" overlayfs
> inside container.
> 
> Anyway, I'm just playing devil's advocate to the idea of two stacked fs
> implementation, so presenting this point of view. I am fully aware that
> there are also plenty of disadvantages to couple two unrelated
> functionalities together.
> 
> Cheers,
> Amir.