All of lore.kernel.org
 help / color / mirror / Atom feed
From: Johannes Pfrang <johannespfrang@gmail.com>
To: linux-btrfs@vger.kernel.org
Subject: Btrfs - distribute files equally across multiple devices
Date: Mon, 6 Jul 2015 18:22:52 +0200	[thread overview]
Message-ID: <559AAB5C.1030001@gmail.com> (raw)

Cross-posting my unix.stackexchange.com question[1] to the btrfs list
(slightly modified):

[1]
https://unix.stackexchange.com/questions/214009/btrfs-distribute-files-equally-across-multiple-devices

---------------------------------------------------------------------------------

I have a btrfs volume across two devices that has metadata RAID 1 and
data RAID 0. AFAIK, in the event one drive would fail, practically all
files above the 64KB default stripe size would be corrupted. As this
partition isn't performance critical, but should be space-efficient,
I've thought about re-balancing the filesystem to distribute files
equally across disks, but something like that doesn't seem to exist. The
ultimate goal would be to be able to still read some of the files in the
event of a drive failure.

AFAIK, using "single"/linear data allocation just fills up drives one by
one (at least that's what the wiki says).

Simple example (according to my best knowledge):

Write two 128KB files (file0, file1) to two devices (dev0, dev1):

RAID0:

    file0/chunk0 (64KB): dev0
    file0/chunk1 (64KB): dev1
    file1/chunk0 (64KB): dev0
    file1/chunk1 (64KB): dev1

Linear:

    file0 (128KB): dev0
    file1 (128KB): dev0

distribute files:

    file0 (128KB): dev0
    file1 (128KB): dev1

The simplest implementation would probably be something like: Always
write files to the disk with the least amount of space used. I think
this may be a valid software-raid use-case, as it combines RAID 0 (w/o
some of the performance gains[2]) with recoverability of about half of
the data/files (balanced by filled space or amount of files) in the
event of a drive-failure[3] by using filesystem information a
hardware-raid doesn't have. In the end this is more or less JBOD with
balanced disk usage + filesystem intelligence.

Is there something like that already in btrfs or could this be something
the btrfs-devs would consider?


[2] Still can read/write multiple files from/to different disks, so less
performance only for "single-file-reads/writes"
[3] using two disks, otherwise (totalDisks-failedDisks)/totalDisks

             reply	other threads:[~2015-07-06 16:23 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-07-06 16:22 Johannes Pfrang [this message]
2015-07-06 16:45 ` Btrfs - distribute files equally across multiple devices Roman Mamedov
2015-07-06 17:31   ` Johannes Pfrang
2015-07-06 17:53 ` Hugo Mills
2015-07-06 18:34   ` Johannes Pfrang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=559AAB5C.1030001@gmail.com \
    --to=johannespfrang@gmail.com \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.