Preparing single-disk setup for future multi-disk usage

Preparing single-disk setup for future multi-disk usage

[Björn Wüst]

Good morning,


I currently have a single-disk setup where I want to use btrfs filesystem. Yet, I expect to add additional disks to this system in the future. Those disks shall be visible to the OS like a single disk, i.e. using multi-disk feature in btrfs. While data shall be striped among those disks in the future, meta data shall be mirrored for better fault tolerance (loss of some data is acceptable, while loss of all data is not acceptable).

btrfs supports multi-disk setups and even adding additional devices at a later point of time. Thus, it is my preferred choice. However, I am puzzled how the mkfs.btrfs command must be parametrized to have RAID1 for meta data and RAID0 for data with just a single disk / partition. Could I simply do mkfs.btrfs -m raid1 -d raid0 /dev/sdaX (where X is the partition number) ?

Unfortunately, I do not have a disk to test it right now. The disk I am planning to use is with the post service still :) . Searching the Web could not reveal a similar scenario. All multi-disk examples assume that you already have multiple disks and not that you are going to have them.

Thank you for your replies to this email (bjoern.wuest@gmx.net, I am not subscribed to the mailing lists, thus please do a 'reply all').


Mit freundlichen Grüßen / Kind regards
  Bjoern Wuest

Re: Preparing single-disk setup for future multi-disk usage

[Fajar A. Nugraha]

On Thu, May 24, 2012 at 1:05 PM, Björn Wüst <Bjoern.Wuest@iteratec.de> wrote:
>
> Unfortunately, I do not have a disk to test it right now. The disk I am planning to use is with the post service still :) .

you can use sparse files. Possibly with losetup, if necessary.

> Thank you for your replies to this email (bjoern.wuest@gmx.net,

That's not the email you use to send

> I am not subscribed to the mailing lists, thus please do a 'reply all').

IMHO asking something to a list and then saying "I am not subscribed"
and "send your reply to this other email address that I'm not using to
send" is rude.

-- 
Fajar

Re: Preparing single-disk setup for future multi-disk usage

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 2245 bytes --]

On Thu, May 24, 2012 at 08:05:23AM +0200, Björn Wüst wrote:
> Good morning,
> 
> 
> I currently have a single-disk setup where I want to use btrfs filesystem. Yet, I expect to add additional disks to this system in the future. Those disks shall be visible to the OS like a single disk, i.e. using multi-disk feature in btrfs. While data shall be striped among those disks in the future, meta data shall be mirrored for better fault tolerance (loss of some data is acceptable, while loss of all data is not acceptable).
> 
> btrfs supports multi-disk setups and even adding additional devices at a later point of time. Thus, it is my preferred choice. However, I am puzzled how the mkfs.btrfs command must be parametrized to have RAID1 for meta data and RAID0 for data with just a single disk / partition. Could I simply do mkfs.btrfs -m raid1 -d raid0 /dev/sdaX (where X is the partition number) ?

   No, this will fail to work, as RAID-1 and -0 both require at least
two disks. What you need to do is use the default settings (mkfs.btrfs
/dev/sdaX) for now, which will give you DUP metadata (two copies of
each block, in different places on the disk), and "single" data (one
copy of the data).

   When you add a new disk, you can convert the replication to
something else, using the balance command:

# btrfs dev add /dev/sdY /mountpoint
# btrfs balance start -dconvert=raid0 /mountpoint

(Note that you don't need to specify -m raid1 to convert the metadata,
as the DUP->RAID-1 conversion is implicit where it's possible).

> Unfortunately, I do not have a disk to test it right now. The disk I am planning to use is with the post service still :) . Searching the Web could not reveal a similar scenario. All multi-disk examples assume that you already have multiple disks and not that you are going to have them.

   The conversion code is quite new, so there's few examples out there
right now.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
  --- There are three things you should never see being made: laws, ---  
                        standards,  and sausages.                        

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

RE: Preparing single-disk setup for future multi-disk usage

[Bjoern Wuest]

Hi Hugo,


Thank you very much for the information.


Regards
  Bjoern


-----Original Message-----
From: Hugo Mills [mailto:hugo@carfax.org.uk] 
Sent: Thursday, May 24, 2012 10:41 AM
To: Björn Wüst
Cc: linux-btrfs@vger.kernel.org
Subject: Re: Preparing single-disk setup for future multi-disk usage

On Thu, May 24, 2012 at 08:05:23AM +0200, Björn Wüst wrote:
> Good morning,
> 
> 
> I currently have a single-disk setup where I want to use btrfs filesystem.
Yet, I expect to add additional disks to this system in the future. Those
disks shall be visible to the OS like a single disk, i.e. using multi-disk
feature in btrfs. While data shall be striped among those disks in the
future, meta data shall be mirrored for better fault tolerance (loss of some
data is acceptable, while loss of all data is not acceptable).
> 
> btrfs supports multi-disk setups and even adding additional devices at a
later point of time. Thus, it is my preferred choice. However, I am puzzled
how the mkfs.btrfs command must be parametrized to have RAID1 for meta data
and RAID0 for data with just a single disk / partition. Could I simply do
mkfs.btrfs -m raid1 -d raid0 /dev/sdaX (where X is the partition number) ?

   No, this will fail to work, as RAID-1 and -0 both require at least two
disks. What you need to do is use the default settings (mkfs.btrfs
/dev/sdaX) for now, which will give you DUP metadata (two copies of each
block, in different places on the disk), and "single" data (one copy of the
data).

   When you add a new disk, you can convert the replication to something
else, using the balance command:

# btrfs dev add /dev/sdY /mountpoint
# btrfs balance start -dconvert=raid0 /mountpoint

(Note that you don't need to specify -m raid1 to convert the metadata, as
the DUP->RAID-1 conversion is implicit where it's possible).

> Unfortunately, I do not have a disk to test it right now. The disk I am
planning to use is with the post service still :) . Searching the Web could
not reveal a similar scenario. All multi-disk examples assume that you
already have multiple disks and not that you are going to have them.

   The conversion code is quite new, so there's few examples out there right
now.

   Hugo.

--
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
  --- There are three things you should never see being made: laws, ---  
                        standards,  and sausages.                        

Re: Preparing single-disk setup for future multi-disk usage

[Calvin Walton]

On Thu, 2012-05-24 at 08:05 +0200, Björn Wüst wrote:
> Good morning,
> 
> 
> I currently have a single-disk setup where I want to use btrfs
> filesystem. Yet, I expect to add additional disks to this system in
> the future. Those disks shall be visible to the OS like a single disk,
> i.e. using multi-disk feature in btrfs. While data shall be striped
> among those disks in the future, meta data shall be mirrored for
> better fault tolerance (loss of some data is acceptable, while loss of
> all data is not acceptable).

If you're using the "RAID 0" (striping) mode, you will not be able to
recover any data in a useful form after a disk failure, because each
file will have been broken into small chunks and spread over both disks.
If you use the 'single' allocation mode instead, it is more likely that
individual files will be mostly on one disk, but there are no
guarantees.

If you want data to be recoverable, you should use a redundant raid
mode; otherwise don't expect that you'll be able to save much after a
disk failure.

-- 
Calvin Walton <calvin.walton@kepstin.ca>