From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dave Chinner Subject: Re: [LSF/FS TOPIC] Scaling file systems on high performance flash devices Date: Wed, 9 Feb 2011 09:38:53 +1100 Message-ID: <20110208223853.GE2559@dastard> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: lsf-pc@lists.linuxfoundation.org, linux-fsdevel@vger.kernel.org To: Nauman Rafique Return-path: Received: from ipmail06.adl6.internode.on.net ([150.101.137.145]:48881 "EHLO ipmail06.adl6.internode.on.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753310Ab1BHWi6 (ORCPT ); Tue, 8 Feb 2011 17:38:58 -0500 Content-Disposition: inline In-Reply-To: Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On Thu, Feb 03, 2011 at 04:39:40PM -0800, Nauman Rafique wrote: > Flash device vendors are coming up with faster and faster devices > every year. Given the high performance supported by these devices, > there are thoughts about using them not only as high performance > storage but also as a replacement for huge quantities of DRAM. That > particular use case would put very stringent requirements on the > performance of file systems on these devices --- an issue that should > be discussed. > > I will share our experience running some experiments on a high > performance flash device (FusionIO IODrive duo) with ext4 and XFS. We > have devised an extensive set of experiments focused on finding the > scaling and overhead problems in the kernel. Our experiments use > various IO sizes, and perform IO in both synchronous multi-threaded > mode and AIO mode. We configure our setup to bypass the block layer > (fusionIO driver supports that), and do IO in O_DIRECT mode to > minimize overhead in the kernel. In spite of such optimizations, we > still see performance issues especially while doing IO at the peak > throughput capacity available on these drives. The issues pertain to > CPU scheduling behavior, filesystem metadata manipulation, and > basically the whole kernel code path involved in doing IO to > such devices, that would not be involved if data was read from DRAM > directly. Seeing as I'm not going to be around for LSF, can you describe some of your testing and the limitations you came across on XFS? I'm especially interested in the metadata manipulation issues you saw as we've done a fair bit of metadata and journal IO optimisation in the past year.... Cheers, Dave. -- Dave Chinner david@fromorbit.com