From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.iobjects.de ([188.40.134.68]:53785 "EHLO mail02.iobjects.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753103AbcD0RXZ (ORCPT ); Wed, 27 Apr 2016 13:23:25 -0400 Subject: Re: Add device while rebalancing To: Juan Alberto Cirez References: <571DFCF2.6050604@gmail.com> <571E154C.9060604@gmail.com> <571F4CD0.9050004@gmail.com> <5720A0E8.5000407@gmail.com> <5720E8FE.2000407@googlemail.com> Cc: linux-btrfs From: =?UTF-8?Q?Holger_Hoffst=c3=a4tte?= Message-ID: <5720F58A.40904@googlemail.com> Date: Wed, 27 Apr 2016 19:23:22 +0200 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Sender: linux-btrfs-owner@vger.kernel.org List-ID: On 04/27/16 18:40, Juan Alberto Cirez wrote: > If this is so, then it leaves even confused. I was under the > impression that the driving imperative for the creation of btrfs was > to address the shortcomings of current filesystems (within the context > of distributed data). That the idea was to create a low level > filesystem that would be the primary choice as a block/brick layer for a > scale-out, distributed data storage... I can't speak for who was or is motivated by what. Btrfs was a necessary reaction to ZFS, and AFAIK this had nothing to do with distributed storage but rather growing concerns around reliability (checksumming), scalability and operational ease: snapshotting, growing/shrinking etc. It's true that some of btrfs' capabilities make it look like a a good candidate, and e.g. Ceph started out using it. For many reasons that didn't work out (AFAIK btrfs maturity + extensibility) - but it also did not address a fundamental mismatch in requirements, which other filesystems (ext4, xfs) could not address either. btrfs simply does "too much" because it has to; you cannot remove or turn off half of what makes a kernel-based filesystem a usable filesystem. This is kind of sad because at its core btrfs *is* an object store with various trees for metadata handling and whatnot - but there's no easy way to turn off all the "Unix is stupid" stuff. AFAIK Gluster will soon also start managing xattrs differently, so this is not limited to Ceph. I've been following this saga for several years now and it's absolutely *astounding* how many bugs and performance problems Ceph has unearthed in existing filesystems, simply because it stresses them in ways they never have been stressed before..only to create the illusion of a distributed key/value store, badly. I don't want to argue about details, you can read more about some of the reasons in [1]. [grumble grumble exokernels and composable things in userland grumble] cheers Holger [1] http://www.slideshare.net/sageweil1/ceph-and-rocksdb