Filesystem hints to storage

Filesystem hints to storage

[Matthew Wilcox]

The NVMHCI working group has specified [1] a set of attributes for
filesystems to pass down to devices which may allow the device to lay
out its storage more effectively (eg if it's a hybrid MLC/SLC device).

For Enterprise NVMHCI, the working group is looking at refining these
hints.  Are there any hints that people would like to see added or
removed from the list in NVMHCI 1.0?

I think there are some hints that are hard for the filesystem to know
itself, never mind pass down, so it'd be good to remove them to reduce
the complexity.  Contrariwise, there are some hints I think the filesystem
could pass down that aren't in the spec today, such as the flags from
madvise (random access vs sequential access) and whether the access is
for fs metadata or application data.

[1] http://www.intel.com/standards/nvmhci/index.htm
Page 35, figure 26

-- 
Matthew Wilcox				Intel Open Source Technology Centre
"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours.  We can't possibly take such
a retrograde step."

Re: Filesystem hints to storage

[Joel Becker]

On Thu, Jul 22, 2010 at 12:08:54PM -0600, Matthew Wilcox wrote:
> For Enterprise NVMHCI, the working group is looking at refining these
> hints.  Are there any hints that people would like to see added or
> removed from the list in NVMHCI 1.0?
> 
> I think there are some hints that are hard for the filesystem to know
> itself, never mind pass down, so it'd be good to remove them to reduce
> the complexity.  Contrariwise, there are some hints I think the filesystem
> could pass down that aren't in the spec today, such as the flags from
> madvise (random access vs sequential access) and whether the access is
> for fs metadata or application data.

	If we're looking at the filesystem interface to all of this, I
would think these hints could be used by non-SSD storage as well.
Not only would a storage array benefit from knowing about random vs
sequential access, but expected redundancy hints would be huge in
managing utilization.

Joel

-- 

Life's Little Instruction Book #396

	"Never give anyone a fruitcake."

Joel Becker
Consulting Software Developer
Oracle
E-mail: joel.becker@oracle.com
Phone: (650) 506-8127

Re: Filesystem hints to storage

[Alan Cox]

> I think there are some hints that are hard for the filesystem to know
> itself, never mind pass down, so it'd be good to remove them to reduce
> the complexity.  Contrariwise, there are some hints I think the filesystem
> could pass down that aren't in the spec today, such as the flags from
> madvise (random access vs sequential access) and whether the access is
> for fs metadata or application data.

Maybe I'm getting cynical with age but I'd put a beer on it being the
case that implementations ignore the fs provided hints within a few years
of it becoming regularly used 8)

Alan

Re: Filesystem hints to storage

[James Bottomley]

On Thu, 2010-07-22 at 20:45 +0100, Alan Cox wrote:
> > I think there are some hints that are hard for the filesystem to know
> > itself, never mind pass down, so it'd be good to remove them to reduce
> > the complexity.  Contrariwise, there are some hints I think the filesystem
> > could pass down that aren't in the spec today, such as the flags from
> > madvise (random access vs sequential access) and whether the access is
> > for fs metadata or application data.
> 
> Maybe I'm getting cynical with age but I'd put a beer on it being the
> case that implementations ignore the fs provided hints within a few years
> of it becoming regularly used 8)

Actually, you're not cynical ... there was actually a paper at fast two
years ago that showed a deductive algorithm would always outperform a
fixed hint list:

http://www.usenix.org/events/fast09/tech/full_papers/liu/liu_html/index.html

The bottom line is that to perform optimally, there's no agreement
between the client and server for what the hints are; the server just
provides a random set of numbers for hints, which the client uses in an
arbitrary fashion and the server processes the hints in different
classes and tries to work out whether they're useful or not.

The beauty is that because there's no fixed agreement, the client is
free to redefine hints as it sees fit (or fashion changes) and the
server will mostly do the best it can anyway.

James