From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andreas Dilger Subject: Re: GFS2/OCFS2 scalability Date: Fri, 20 Feb 2009 12:50:30 -0700 Message-ID: <20090220195030.GE3199@webber.adilger.int> References: <2d42915c0902200923u6c539fbdh3ec1a147b4daa396@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7BIT Cc: linux-fsdevel@vger.kernel.org, ocfs2-users@oss.oracle.com, linux-cluster@redhat.com To: Kirill Kuvaldin Return-path: Received: from sca-es-mail-2.Sun.COM ([192.18.43.133]:61599 "EHLO sca-es-mail-2.sun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756510AbZBTTvJ (ORCPT ); Fri, 20 Feb 2009 14:51:09 -0500 Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n1KJp4ZP021101 for ; Fri, 20 Feb 2009 11:51:06 -0800 (PST) Content-disposition: inline Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7.0-3.01 64bit (built Dec 23 2008)) id <0KFD00C00PS8OA00@fe-sfbay-09.sun.com> for linux-fsdevel@vger.kernel.org; Fri, 20 Feb 2009 11:32:09 -0800 (PST) In-reply-to: <2d42915c0902200923u6c539fbdh3ec1a147b4daa396@mail.gmail.com> Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On Feb 20, 2009 20:23 +0300, Kirill Kuvaldin wrote: > I'm evaluating different cluster file systems that can work with large > clustered environment, e.g. hundreds of nodes connected to a SAN over > FC. > > > So far I looked at OCFS2 and GFS2, they both worked nearly the same > in terms of performance, but since I ran my tests in a local limited > environment with 4 nodes in a cluster, the results can't be > extrapolated to 100 nodes. > > Is there any reading on comparison of OCFS2/GFS2 scalability? And > could these FS scale up to environment of that size (100-1000 nodes, > 10-100Tb storage) or should I look to proprietary solutions like IBM > GPFS, HP Polyserve, etc... ? I'm fairly biased, but I think you should look at Lustre - it is currently running on 15 of the 20 fastest systems in the world, and 40% of the top 200. No problems with scalability at all - the ORNL Jaguar system has 37000+ nodes, 10PB of storage, and has exceeded 150GB/s peak read/write speed. Lustre is GPL and freely available: http://www.lustre.org/ Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.