btrfs (general) raid for other filesystems?

btrfs (general) raid for other filesystems?

[Martin]

Just a random Sunday afternoon thought:

We've got some rather nice variations on the block-level RAID schemes
but instead being implemented at the filesystem level in btrfs...

Could the btrfs RAID be coded to be general so that a filesystem stack
could be set up whereby the filesystem level raids could be used for ANY
filesystem?


So for example, we could have the stack:


filesystem level RAID

 |
 V

filesystem

 |
 V

Block level


So, an interesting variation could be to have filesystem level raid
operating on ext4 or nilfs or whatever... Would that be a sensible idea?



Regards,
Martin

Re: btrfs (general) raid for other filesystems?

[Clemens Eisserer]

Hi Martin,

> So, an interesting variation could be to have filesystem level raid
> operating on ext4 or nilfs or whatever... Would that be a sensible idea?

Thats already supported by using LVM. What do you think you would gain
from layering in top of btrfs?

- Clemens

Re: btrfs (general) raid for other filesystems?

[Martin]

On 19/05/13 18:39, Clemens Eisserer wrote:
> Hi Martin,
> 
>> So, an interesting variation could be to have filesystem level raid
>> operating on ext4 or nilfs or whatever... Would that be a sensible idea?
> 
> Thats already supported by using LVM. What do you think you would gain
> from layering in top of btrfs?

md-raid and lvm-raid are raid at the block level.

btrfs-raid offers a greater variety and far greater flexibility of raid
options individually for filedata and metadata at the filesystem level.

raid at the filesystem level should also gain higher performance over
that of just blindly replicating blocks of binary data across devices at
the block level.


My thoughts are to take advantage of the btrfs-raid work being done but
for all filesystems. Hence, we can then have a very flexible raid
available for whatever filesystem might be best for any underlying device.

OK... So we make all of lvm, md-raid, and drbd all redundant!

Regards,
Martin

Re: btrfs (general) raid for other filesystems?

[Chris Murphy]

On May 19, 2013, at 12:59 PM, Martin <m_btrfs@ml1.co.uk> wrote:
> 
> btrfs-raid offers a greater variety and far greater flexibility of raid
> options individually for filedata and metadata at the filesystem level.

Well it really doesn't. The btrfs raid advantages leverage prior work that makes btrfs what it is.


> OK... So we make all of lvm, md-raid, and drbd all redundant!

No they are different things for different use cases. What you seem to be asking for is for a ZFS-like feature that allows other file systems to exist on ZFS, and thereby gaining some of the advantage of the underlying file system.


Chris Murphy

Re: btrfs (general) raid for other filesystems?

[Martin]

On 19/05/13 20:34, Chris Murphy wrote:
> On May 19, 2013, at 12:59 PM, Martin <m_btrfs@ml1.co.uk> wrote:
>> 
>> btrfs-raid offers a greater variety and far greater flexibility of
>> raid options individually for filedata and metadata at the
>> filesystem level.
> 
> Well it really doesn't. The btrfs raid advantages leverage prior work
> that makes btrfs what it is.

Indeed, the "btrfs raid" as evolving looks to be tightly part of btrfs
itself, shaped by what is being done in btrfs...

And also there is the work going into how the 'raid' semantics
operate for data and the filesystem metadata.

Also tied into that is storage balancing and load (io bandwidth)
balancing with most recently developers looking how to move 'hot' data
onto preferred physical drives?


>> OK... So we make all of lvm, md-raid, and drbd all redundant!
> 
> No they are different things for different use cases. What you seem
> to be asking for is for a ZFS-like feature that allows other file
> systems to exist on ZFS, and thereby gaining some of the advantage of
> the underlying file system.

That's going a little too deep...


My thoughts are much more shallow. Can the raid and load balancing work
being done for btrfs be bundled up so as to permit that to also be used
instead as a filesystem layer that then utilises /any/ underlying
filesystem?

So, instead of btrfs style file-level raid and load balancing only on
devices which have been formatted with btrfs, the raid and load
balancing operates as a filsystem layer that coordinates storing files
on any motley collection of multiple whatever filesystem-on-device.

Obvious enough for raid1 to 'tee' a file out to multiple filesystems.

For the other raids, filenames would need to be munged to denote their
multiple parts (simply always append a 6 character index?). raid0 would
need a file to be split into parts and then those file parts
concatenated upon reading under the original filename. raid5/6 would
similarly need file splitting but also the data redundancy added.

For example, for paranoid redundancy and fast operation:


raid1 + load balance

 |
 V

btrfs on HDD1, ext4 on HDD2, NILFS on flash1, nfs to host2


Obviously, doing that loses any features (such as snapshots) not common
across all the group.


As for a use case? Would that be a good idea or not? :-)

One thought is that users could set up funky redundant operation across
networked devices using nfs.

Another thought is that we go to an awful lot of trouble to accommodate
extremely different storage technologies that are only ever going to
physically diverge further. For example, we have HDDs and SSDs. We also
have much cheaper flash with very limited wear levelling ideal for
'cold' data. Or even raw flash without all the proprietary firmware
obscurity... Hence dedicate a particular filesystem to each rather than
one monster for all?


The "raid + load balance" could be a well defined layer with no or few
special hooks into the lower layers.


All just a thought...

Regards,
Martin