Re: Inode and dentry cache behavior

[Dave Chinner <david@fromorbit.com>]

On Mon, May 11, 2015 at 02:07:39PM -0700, Shrinand Javadekar wrote:
> In case you're curious, the solutions to this problem are being
> discussed at [1] on the Swift side.
> 
> O_TMPFILE and linkat() aren't available in all kernels (and python)
> and therefore not a viable option right away. Among few others, one of
> the proposals is to shard the tmp directory into say 256 dirs and
> create these files inside those dirs. That way files won't get created
> in a single AG. Are there any other problems if this is done?

Yes, you don't get any locality between files in the same directory
once you move them to their final location in the hash tree. You'll
end up with the same problems you are trying to solve now as the
object count goes up - you'll simply have taken a different path to
the same destination....

Create your temp files in their eventual destination directory -
doing anything else is pretty much guaranteed to compromise all the
locality algorithms the filesystem uses. Without good locality
within the data set, performance of lookups and writeback will
degrade to large seek time random IO pretty quickly.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs