Re: [LSF/FS TOPIC] Scaling file systems on high performance flash devices

[Dave Chinner <david@fromorbit.com>]

On Thu, Feb 03, 2011 at 04:39:40PM -0800, Nauman Rafique wrote:
> Flash device vendors are coming up with faster and faster devices
> every year.  Given the high performance supported by these devices,
> there are thoughts about using them not only as high performance
> storage but also as a replacement for huge quantities of DRAM. That
> particular use case would put very stringent requirements on the
> performance of file systems on these devices --- an issue that should
> be discussed.
> 
> I will share our experience running some experiments on a high
> performance flash device (FusionIO IODrive duo) with ext4 and XFS. We
> have devised an extensive set of experiments focused on finding the
> scaling and overhead problems in the kernel. Our experiments use
> various IO sizes, and perform IO in both synchronous multi-threaded
> mode and AIO mode. We configure our setup to bypass the block layer
> (fusionIO driver supports that), and do IO in O_DIRECT mode to
> minimize overhead in the kernel. In spite of such optimizations, we
> still see performance issues especially while doing IO at the peak
> throughput capacity available on these drives. The issues pertain to
> CPU scheduling behavior, filesystem metadata manipulation, and
> basically the whole kernel code path involved in doing IO to
> such devices, that would not be involved if data was read from DRAM
> directly.

Seeing as I'm not going to be around for LSF, can you describe some
of your testing and the limitations you came across on XFS? I'm
especially interested in the metadata manipulation issues you saw as
we've done a fair bit of metadata and journal IO optimisation in the
past year....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com