[RFC] remove filestreams support?

[RFC] remove filestreams support?

[Christoph Hellwig]

I'd like to know if there are any people actively using the filestreams
support in XFS (-o filestreams).  It's pretty much a fringe feature, and
I've not seen any user reports for it, while it never passes all its
XFSQA tests reliably.  It was added specificly for CXFS media streaming
operations on one particular array model that hasn't been sold for a
long time.  The feature purely is an in-memory one so unlike for example
the realtime device there are no issue about beeing able to read old
filesystems.  The filestreams specific files are around 1500 lines of
code, not even counting the hooks in the core XFS codebase.

If no serious users reply to this mail I'd like to add a deprecation
warning to the mount option in the Linux 3.0 release cycle, with a goal
to drop it 4 releases later.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [RFC] remove filestreams support?

[Stewart Smith]

On Mon, 4 Jul 2011 11:34:43 -0400, Christoph Hellwig <hch@infradead.org> wrote:
> I'd like to know if there are any people actively using the filestreams
> support in XFS (-o filestreams).  It's pretty much a fringe feature, and
> I've not seen any user reports for it, while it never passes all its
> XFSQA tests reliably.  It was added specificly for CXFS media streaming
> operations on one particular array model that hasn't been sold for a
> long time.  The feature purely is an in-memory one so unlike for example
> the realtime device there are no issue about beeing able to read old
> filesystems.  The filestreams specific files are around 1500 lines of
> code, not even counting the hooks in the core XFS codebase.

I used it a bit on MythTV box to help circumvent the stupid behaviour of
it fsync()ing every second[1] leading to poor file layout on disk. I
found that filestreams worked a bit better than just setting allocsize
mount option... but I wouldn't be screaming too much if it went away....


[1] which was to work around ext3 taking locks for so long that you'd
then miss part of your TV program

-- 
Stewart Smith

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [RFC] remove filestreams support?

[Dave Chinner]

On Mon, Jul 04, 2011 at 11:34:43AM -0400, Christoph Hellwig wrote:
> I'd like to know if there are any people actively using the filestreams
> support in XFS (-o filestreams).  It's pretty much a fringe feature, and
> I've not seen any user reports for it, while it never passes all its
> XFSQA tests reliably.  It was added specificly for CXFS media streaming
> operations on one particular array model that hasn't been sold for a
> long time.  The feature purely is an in-memory one so unlike for example
> the realtime device there are no issue about beeing able to read old
> filesystems.  The filestreams specific files are around 1500 lines of
> code, not even counting the hooks in the core XFS codebase.
> 
> If no serious users reply to this mail I'd like to add a deprecation
> warning to the mount option in the Linux 3.0 release cycle, with a goal
> to drop it 4 releases later.

I'm not so concerned about the actual filestreams allocator - it
makes a fair bit of mess through the allocator logic that I've always
disliked. However, there are bits of it that could be useful in
future, and if done right we could retain the filestreams allocator
functionality for those that need it.

That is, the act of assigning an inode or group of inodes to an
allocation group for the express purpose of providing locality of
allocation is useful. I've been looking at using this functionality
for cgroup-aware allocation. In that case, different cgroups would
be assigned different AGs to keep them logically (and potentially
physically) separate, and inodes that are dirtied by a process in
a cgroups would then be associated with the assigned AG.

In fact, I'd suggest that this makes a more sensible method of
implementing a filestreams policy, because simply placing the
competing stream writer processes into separate cgroups would have
the same effect as the current "associate all the files in a
directory our process group is about to create with the same AG"
policy without needing on-disk flags or mount options to trigger
that behaviour.

The worst of the filestreams implementation is the
code needed to handle the potential locking inversions of also
having to get the parent directory inode during allocation to set up
the association, and this would no longer be needed.

It also gets around the problem of having to maintain and time out
associations via reference counts as well (the whole mru cache
thingy), because the cgroup association can be looked up from the
inode whenever it is needed, including during writeback when doing
delayed allocation (dependent on cgroup-aware flusher
infrastructure, but that is in progress). This greatly simplfies the
code that we'd need to maintain as well....

So rather than deprecating the functionality, perhaps we should look
at implementing it through a simpler, more generic, better
integrated interface? That will increase the usefulness of the
functionality for a much wider audience than it has now, and also
provide the virt/blk throttling folk with exactly the "don't cross
the streams" functionality they suggest filesystems are unable to
support easily.....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [RFC] remove filestreams support?

[Christoph Hellwig]

On Tue, Jul 05, 2011 at 11:54:58AM +1000, Dave Chinner wrote:
> So rather than deprecating the functionality, perhaps we should look
> at implementing it through a simpler, more generic, better
> integrated interface? That will increase the usefulness of the
> functionality for a much wider audience than it has now, and also
> provide the virt/blk throttling folk with exactly the "don't cross
> the streams" functionality they suggest filesystems are unable to
> support easily.....

That does indeed sound simpler, and also more useful.  Do the users
who have chimed in here (and off list) think such a scheme would be
useful for them?

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [RFC] remove filestreams support?

[Alex Elder]

On Tue, 2011-07-05 at 07:01 -0400, Christoph Hellwig wrote:
> On Tue, Jul 05, 2011 at 11:54:58AM +1000, Dave Chinner wrote:
> > So rather than deprecating the functionality, perhaps we should look
> > at implementing it through a simpler, more generic, better
> > integrated interface? That will increase the usefulness of the
> > functionality for a much wider audience than it has now, and also
> > provide the virt/blk throttling folk with exactly the "don't cross
> > the streams" functionality they suggest filesystems are unable to
> > support easily.....
> 
> That does indeed sound simpler, and also more useful.  Do the users
> who have chimed in here (and off list) think such a scheme would be
> useful for them?

We have customers that make good use of filestreams.
I don't think it matters whether they use CXFS or XFS,
it's a very useful allocation strategy, and an important
feature for certain applications.  I like the idea of
generalizing it and/or improving its interface though.

					-Alex


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs