Re: Streams support in Linux

Matthew Wilcox
2018-08-26T00:37:22Z

On Sat, Aug 25, 2018 at 07:00:26PM +0100, Al Viro wrote:
> > > Where do dentries of these things belong, anyway?
> >
> > Do streams have to have dentries? They don't have names in the hierarchy.
> > Can they share their parent's dentry? Can they have a NULL dentry? Or
> > a magic dentry of some kind?
>
> If you want them as opened files - yes, they have to, no, they can't and no,
> they can't.

I was hoping for something like alloc_file_pseudo() rather than having
real dentries in the dentry tree.

> > > > unlinkat()
> > > > This is how you remove an individual named stream
> > >
> > > Yeah... and what happens to access via other hardlinks to the same parent
> > > file?
> >
> > I'm not sure I understand the problem here. You have, say /a/b/c
> > and /d which refer to the same inode. This inode has streams x and y.
> > If you unlinkat() /a/b/c::x then it is no longer openable. Any currently
> > existing open fds for x keep x alive until the last one is closed, no
> > matter which path was used to open them.
>
> Again, where it the tree would the dentries live? And I really hope you are
> not suggesting to reserve '::' - not when we have files like
> /usr/share/man/man3/Params::Check.3perl.gz
> just to pull a random line out of locate :: output here...

I was just using '::' as shorthand for this email. Just to be clear,
there's no intent to have these streams in the filesystem namespace.

> > > > Unlinking a file with named streams removes all named streams from that
> > > > file and then unlinks the file. Open streams will continue to exist in
> > > > the filesystem until they are closed, just as unlinked files do.
> > >
> > > Can they be looked up by openat() from such a starting point?
> >
> > No, you can't use openat() on a stream fd. I should have been clearer
> > on that above; this proposal should have split the syscall list in two;
> > one for how the syscall behaves on a fd-for-a-file-with-streams and one
> > for how the syscall behaves on a stream-fd.
>
> I'm not suggesting passing a stream fd as the starting point; just a normal
> opened-and-then-unlinked regular file.

Ah, I see your point. I think that it makes most sense to define the
semantics of an open-but-unlinked file that you can create new streams,
delete existing streams and rename streams until the last reference to
that file are gone, and all references to streams within a file count
as a reference on that file. Does that make sense?

> > Right, but that code also assumes it's dealing with files.
>
> I'm looking forward to your analysis of all code related to dentry tree,
> with an eye towards the places where such asserts are quietly made...

I really was hoping not to change the dentry tree at all.