Re: Streams support in Linux

[Jeremy Allison <jra@samba.org>]

On Mon, Aug 27, 2018 at 08:07:35PM -0500, Steve French wrote:
> On Mon, Aug 27, 2018 at 7:45 PM Theodore Y. Ts'o <tytso@mit.edu> wrote:
> >
> > On Mon, Aug 27, 2018 at 11:21:43AM -0700, Matthew Wilcox wrote:
> > > On Mon, Aug 27, 2018 at 10:05:31AM -0700, Jeremy Allison wrote:
> > > > I can't think of a *single* case where a stream adds more
> > > > utility than an EA used in the same case.
> > > >
> > > > I don't want theoretical "well it would be nice if..",
> > > > I want clear "we couldn't have done it any other way"
> > > > kinds of things.
> > >
> > > I started this thread with such an example.  The fs-verity patch proposed
> > > wants to store hundreds of megabytes of data associated with a particular
> > > file.  The current solution is to append it to the end of the data then
> > > magic to set i_size lower but not remove the data from the file like a
> > > truncate would.  Then more magic to read that data.
> >
> > Sure, but so what?  The Merkle Tree only needs to be read by the
> > kernel, and we only need to install the fs-verity data once.  And
> > putting it at the end of the file works just fine.  Theoretically we
> > could do it other ways, but this is almost certainly easier and
> > cleaner than if we had to open a streams file from kernel code.
> >
> > Doing it using streams would actually be *harder* and is only of use
> > for people who like more structured designs.  But to be clear, this is
> > *not* something I'm asking for as one of the original designers of
> > fs-verity.
> 
> Given that streams need to be read to backup Macs and Windows
> (and for a few features of these servers mentioned earlier)
> and would be exposed in ntfs (and SMB3 remotely) locally on Linux,
> seems useful to me to have some consistent way to open and read
> them on Linux even if we don't want to generalize it to other local fs.
> The protocol supports it fine (just a file with a reserved character
> ':' in it) but a little tricky to avoid name conflict with posix ':'
> in filenames

This sounds like a case for a couple of ioctls. One to enumerate
the streams on an open fd, one to open a given stream name on an
open fd.