From mboxrd@z Thu Jan 1 00:00:00 1970 From: Steve Barber Subject: Re: Using locally replicated OSDs to reduce Ceph replication Date: Wed, 17 Apr 2013 19:02:00 -0400 Message-ID: <20130417230200.GC4565@nist.gov> References: <20130417220251.GZ4565@nist.gov> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Return-path: Received: from wsget1.nist.gov ([129.6.13.150]:7142 "EHLO wsget1.nist.gov" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S966936Ab3DQXCF (ORCPT ); Wed, 17 Apr 2013 19:02:05 -0400 Content-Disposition: inline In-Reply-To: Sender: ceph-devel-owner@vger.kernel.org List-ID: To: Gregory Farnum Cc: Jeff Mitchell , ceph-devel On Wed, Apr 17, 2013 at 06:23:43PM -0400, Gregory Farnum wrote: > On Wed, Apr 17, 2013 at 3:02 PM, I wrote: > > In particular, has anyone tried making a big RAID set (of any type) and > > carving out space (logical volumes, zvols, etc.) to become virtual OSDs? > > Any architectural gotchas with this idea? > > I believe there are some people running with this architecture; > there's just less knowledge about how it behaves in the long term. It > should be fine subject to the standard issues with RAID5/6 small > writes, which OSDs do a lot of (and I don't know why you'd bother > using a mirroring RAID instead of Ceph replication!). > I can say that there would be little point to carving up the arrays > into multiple OSDs; other than that, have fun. :) > -Greg > Software Engineer #42 @ http://inktank.com | http://ceph.com So you think it'd be worth trying just running a few really large OSDs in a configuration like that? I wasn't sure that would scale as well, but I'm still pretty new to Ceph. About mirroring/RAID vs. Ceph replication, I was under the impression that there would be a lot of extra network traffic generated by writes with so many replicas which might not be optimal. True enough about RAID 5/6 small writes. Just gotta try it and see I guess. Thanks for the feedback! Steve