btrfs raid assurance

btrfs raid assurance

[Hérikz Nawarro]

Hello everyone,

I'm migrating to btrfs and i would like to know, in a btrfs filesystem
with 4 disks (multiple sizes) with -d raid0 & -m raid1, how many
drives can i lost without losing the entire array?

Cheers.

Re: btrfs raid assurance

[Austin S. Hemmelgarn]

On 2017-07-25 08:55, Hérikz Nawarro wrote:
> Hello everyone,
> 
> I'm migrating to btrfs and i would like to know, in a btrfs filesystem
> with 4 disks (multiple sizes) with -d raid0 & -m raid1, how many
> drives can i lost without losing the entire array?
Exactly one, but you will lose data if you lose one device.  Most of the 
BTRFS profiles are poorly named (single and dup being the exceptions), 
and do not behave consistently with the RAID levels they are named 
after.  BTRFS raid1 mode is functionally equivalent to MD or LVM RAID10 
mode, just with larger blocks.  It only gives you two copies of the 
data, so losing one device will make the array degraded, and two will 
effectively nuke it.

If you care about data safety, I would advise against using raid0 for 
data.  If you lose _one_ device in raid0 mode, you will usually lose 
part of most of the files on the volume.  Single mode for data will 
still distribute things evenly and will not have that issue (unless you 
have files larger than 1GB, a file will either be all there or all gone, 
as opposed to having read errors part way through), and isn't much worse 
in terms of performance (BTRFS does not parallelize device access as 
well as it should).

If you care about both performance and data safety, and can tolerate 
having only half the usable space, I would actually suggest running 
BTRFS with both data and metadata in raid1 mode on top of two LVM or MD 
RAID0 volumes.  This should outperform the configuration you listed by a 
significant amount, will provide better data safety, and should also do 
a better job of distributing the load across devices.

Re: btrfs raid assurance

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 1477 bytes --]

On Tue, Jul 25, 2017 at 09:55:37AM -0300, Hérikz Nawarro wrote:
> Hello everyone,
> 
> I'm migrating to btrfs and i would like to know, in a btrfs filesystem
> with 4 disks (multiple sizes) with -d raid0 & -m raid1, how many
> drives can i lost without losing the entire array?

   You can lose one device in the array, and the FS structure will be
OK -- it will still mount, and you'll be able to see all the filenames
and directory structures and so on.

   However, if you do lose one device, then you'll lose
(approximately) half of the bytes in all of your files, most likely in
alternating 64k slices in each file. Attempting to read the missing
parts will result in I/O errors being returned from the filesystem.

   So, while the FS is in theory still fine as a (probably read-only)
filesystem, it's actually going to be *completely* useless with a
missing device, because none of your file data will be usably intact.

   If you want the FS to behave well when you lose a device, you'll
need some kind of actual redundancy in the data storage part -- RAID-1
would be my recommendation (it stores two copies of each piece of
data, so you can lose up to one device and still be OK).

   Hugo.

-- 
Hugo Mills             | One of these days, I'll catch that man without a
hugo@... carfax.org.uk | quotation, and he'll look undressed.
http://carfax.org.uk/  |
PGP: E2AB1DE4          |                                   Leto Atreides, Dune

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: btrfs raid assurance

[Hérikz Nawarro]

Thanks everyone, I'll stick with raid 1.

Re: btrfs raid assurance

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 1261 bytes --]

On Tue, Jul 25, 2017 at 01:46:56PM +0000, Hugo Mills wrote:
> On Tue, Jul 25, 2017 at 09:55:37AM -0300, Hérikz Nawarro wrote:
> > Hello everyone,
> > 
> > I'm migrating to btrfs and i would like to know, in a btrfs filesystem
> > with 4 disks (multiple sizes) with -d raid0 & -m raid1, how many
> > drives can i lost without losing the entire array?

   Oh, and one other thing -- if you have different-sized devices,
RAID-0 is probably the wrong thing to be using anyway, as you won't be
able to use the difference between the largest and second-largest
device. If you want to use all the space on the available devices,
then "single" mode is probably better (although you still lose a lot
of data if a device breaks), or RAID-1 (which will cope well with the
different sizes as long as the largest device is smaller than the rest
of them added together).

   You can see about the disk usage in different scenarios with the
online tool at:

http://carfax.org.uk/btrfs-usage/

   Hugo.

-- 
Hugo Mills             | One of these days, I'll catch that man without a
hugo@... carfax.org.uk | quotation, and he'll look undressed.
http://carfax.org.uk/  |
PGP: E2AB1DE4          |                                   Leto Atreides, Dune

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: btrfs raid assurance

[Hérikz Nawarro]

And btw, my current disk conf is a 1x 500GB, 2x3TB and a 5TB.

2017-07-25 10:51 GMT-03:00 Hugo Mills <hugo@carfax.org.uk>:
> On Tue, Jul 25, 2017 at 01:46:56PM +0000, Hugo Mills wrote:
>> On Tue, Jul 25, 2017 at 09:55:37AM -0300, Hérikz Nawarro wrote:
>> > Hello everyone,
>> >
>> > I'm migrating to btrfs and i would like to know, in a btrfs filesystem
>> > with 4 disks (multiple sizes) with -d raid0 & -m raid1, how many
>> > drives can i lost without losing the entire array?
>
>    Oh, and one other thing -- if you have different-sized devices,
> RAID-0 is probably the wrong thing to be using anyway, as you won't be
> able to use the difference between the largest and second-largest
> device. If you want to use all the space on the available devices,
> then "single" mode is probably better (although you still lose a lot
> of data if a device breaks), or RAID-1 (which will cope well with the
> different sizes as long as the largest device is smaller than the rest
> of them added together).
>
>    You can see about the disk usage in different scenarios with the
> online tool at:
>
> http://carfax.org.uk/btrfs-usage/
>
>    Hugo.
>
> --
> Hugo Mills             | One of these days, I'll catch that man without a
> hugo@... carfax.org.uk | quotation, and he'll look undressed.
> http://carfax.org.uk/  |
> PGP: E2AB1DE4          |                                   Leto Atreides, Dune

Re: btrfs raid assurance

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 1740 bytes --]

On Tue, Jul 25, 2017 at 10:55:18AM -0300, Hérikz Nawarro wrote:
> And btw, my current disk conf is a 1x 500GB, 2x3TB and a 5TB.

   OK, so by my mental arithmetic(*), you'd get:

 -  9.5  TB usable in RAID-0
 - 11.5  TB usable in single mode
 -  5.75 TB usable in RAID-1

   Hugo.

(*) Which may be a bit wobbly. :)

> 2017-07-25 10:51 GMT-03:00 Hugo Mills <hugo@carfax.org.uk>:
> > On Tue, Jul 25, 2017 at 01:46:56PM +0000, Hugo Mills wrote:
> >> On Tue, Jul 25, 2017 at 09:55:37AM -0300, Hérikz Nawarro wrote:
> >> > Hello everyone,
> >> >
> >> > I'm migrating to btrfs and i would like to know, in a btrfs filesystem
> >> > with 4 disks (multiple sizes) with -d raid0 & -m raid1, how many
> >> > drives can i lost without losing the entire array?
> >
> >    Oh, and one other thing -- if you have different-sized devices,
> > RAID-0 is probably the wrong thing to be using anyway, as you won't be
> > able to use the difference between the largest and second-largest
> > device. If you want to use all the space on the available devices,
> > then "single" mode is probably better (although you still lose a lot
> > of data if a device breaks), or RAID-1 (which will cope well with the
> > different sizes as long as the largest device is smaller than the rest
> > of them added together).
> >
> >    You can see about the disk usage in different scenarios with the
> > online tool at:
> >
> > http://carfax.org.uk/btrfs-usage/
> >
> >    Hugo.
> >

-- 
Hugo Mills             | One of these days, I'll catch that man without a
hugo@... carfax.org.uk | quotation, and he'll look undressed.
http://carfax.org.uk/  |
PGP: E2AB1DE4          |                                   Leto Atreides, Dune

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: btrfs raid assurance

[waxhead]

Hugo Mills wrote:
>
>>>     You can see about the disk usage in different scenarios with the
>>> online tool at:
>>>
>>> http://carfax.org.uk/btrfs-usage/
>>>
>>>     Hugo.
>>>
As a side note, have you ever considered making this online tool (that 
should never go away just for the record) part of btrfs-progs e.g. a 
proper tool? I use it quite often (at least several timers per. month) 
and I would love for this to be a visual tool 'btrfs-space-calculator' 
would be a great name for it I think.

Imagine how nice it would be to run....

btrfs-space-calculator -mraid1 -draid10 /dev/sda1 /dev/sdb1 /dev/sdc2 
/dev/sdd2 /dev/sde3 for example and instantly get something similar to 
my example below (no accuracy intended)

d=data
m=metadata
.=unusable

{  500mb} [|ddddd|] /dev/sda1
{ 3000mb} [|ddddd|mmmmm|mmmmm|mmmmm|mmmmm|mm...|] /dev/sdb1
{ 3000mb} [|ddddd|mmmmm|mmmmm|mmmmm|mmmmm|mmm..|] /dev/sdc2
{ 5000mb} 
[|ddddd|mmmmm|mmmmm|mmmmm|mmmmm|mmmmm|mmmmm|mmmmm|mmmmm|mmmmm|] /dev/sdb1
--------
{11500mb} Total space

usable for data (raid10): 1000mb / 2000mb
usable for metadata (raid1): 4500mb / 9000mb
unusable: 500mb

Of course this would have to change one (if ever) subvolumes can have 
different raid levels etc, but I would have loved using something like 
this instead of jumping around carfax abbey (!) at night.

Re: btrfs raid assurance

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 2515 bytes --]

On Tue, Jul 25, 2017 at 11:29:13PM +0200, waxhead wrote:
> 
> 
> Hugo Mills wrote:
> >
> >>>    You can see about the disk usage in different scenarios with the
> >>>online tool at:
> >>>
> >>>http://carfax.org.uk/btrfs-usage/
> >>>
> >>>    Hugo.
> >>>
> As a side note, have you ever considered making this online tool
> (that should never go away just for the record) part of btrfs-progs
> e.g. a proper tool? I use it quite often (at least several timers
> per. month) and I would love for this to be a visual tool
> 'btrfs-space-calculator' would be a great name for it I think.
> 
> Imagine how nice it would be to run....
> 
> btrfs-space-calculator -mraid1 -draid10 /dev/sda1 /dev/sdb1
> /dev/sdc2 /dev/sdd2 /dev/sde3 for example and instantly get
> something similar to my example below (no accuracy intended)

   It's certainly a thought. I've already got the algorithm written
up. I'd have to resurrect my C skills, though, and it's a long way
down my list of things to do. :/

   Also on the subject of this tool, I'd like to make it so that the
parameters get set in the URL, so that people can copy-paste the URL
of the settings they've got into IRC for discussion. However, that
would involve doing more JavaScript, which is possibly even lower down
my list of things to do than starting doing C again...

   Hugo.

> d=data
> m=metadata
> .=unusable
> 
> {  500mb} [|ddddd|] /dev/sda1
> { 3000mb} [|ddddd|mmmmm|mmmmm|mmmmm|mmmmm|mm...|] /dev/sdb1
> { 3000mb} [|ddddd|mmmmm|mmmmm|mmmmm|mmmmm|mmm..|] /dev/sdc2
> { 5000mb}
> [|ddddd|mmmmm|mmmmm|mmmmm|mmmmm|mmmmm|mmmmm|mmmmm|mmmmm|mmmmm|]
> /dev/sdb1
> --------
> {11500mb} Total space
> 
> usable for data (raid10): 1000mb / 2000mb
> usable for metadata (raid1): 4500mb / 9000mb
> unusable: 500mb
> 
> Of course this would have to change one (if ever) subvolumes can
> have different raid levels etc, but I would have loved using
> something like this instead of jumping around carfax abbey (!) at
> night.

   The core algorithm for the tool actually works pretty well for
dealing with different RAID levels, as long as you know how much of
each kind of data you're going to be using. (Although it's actually
path-dependent -- write 100 GB of RAID-0 then 100 GB of RAID-1 can
have different results than if you write them in the opposite order --
but that's a kind of edge effect).

   Hugo.

-- 
Hugo Mills             | Great oxymorons of the world, no. 4:
hugo@... carfax.org.uk | Future Perfect
http://carfax.org.uk/  |
PGP: E2AB1DE4          |

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: btrfs raid assurance

[Austin S. Hemmelgarn]

On 2017-07-25 17:45, Hugo Mills wrote:
> On Tue, Jul 25, 2017 at 11:29:13PM +0200, waxhead wrote:
>>
>>
>> Hugo Mills wrote:
>>>
>>>>>     You can see about the disk usage in different scenarios with the
>>>>> online tool at:
>>>>>
>>>>> http://carfax.org.uk/btrfs-usage/
>>>>>
>>>>>     Hugo.
>>>>>
>> As a side note, have you ever considered making this online tool
>> (that should never go away just for the record) part of btrfs-progs
>> e.g. a proper tool? I use it quite often (at least several timers
>> per. month) and I would love for this to be a visual tool
>> 'btrfs-space-calculator' would be a great name for it I think.
>>
>> Imagine how nice it would be to run....
>>
>> btrfs-space-calculator -mraid1 -draid10 /dev/sda1 /dev/sdb1
>> /dev/sdc2 /dev/sdd2 /dev/sde3 for example and instantly get
>> something similar to my example below (no accuracy intended)
> 
>     It's certainly a thought. I've already got the algorithm written
> up. I'd have to resurrect my C skills, though, and it's a long way
> down my list of things to do. :/
> 
>     Also on the subject of this tool, I'd like to make it so that the
> parameters get set in the URL, so that people can copy-paste the URL
> of the settings they've got into IRC for discussion. However, that
> would involve doing more JavaScript, which is possibly even lower down
> my list of things to do than starting doing C again...
Is the core logic posted somewhere?  Because if I have some time, I 
might write up a quick Python script to do this locally (it may not be 
as tightly integrated with the regular tools, but I can count on half a 
hand how many distros don't include Python by default).
> 
>     Hugo.
> 
>> d=data
>> m=metadata
>> .=unusable
>>
>> {  500mb} [|ddddd|] /dev/sda1
>> { 3000mb} [|ddddd|mmmmm|mmmmm|mmmmm|mmmmm|mm...|] /dev/sdb1
>> { 3000mb} [|ddddd|mmmmm|mmmmm|mmmmm|mmmmm|mmm..|] /dev/sdc2
>> { 5000mb}
>> [|ddddd|mmmmm|mmmmm|mmmmm|mmmmm|mmmmm|mmmmm|mmmmm|mmmmm|mmmmm|]
>> /dev/sdb1
>> --------
>> {11500mb} Total space
>>
>> usable for data (raid10): 1000mb / 2000mb
>> usable for metadata (raid1): 4500mb / 9000mb
>> unusable: 500mb
>>
>> Of course this would have to change one (if ever) subvolumes can
>> have different raid levels etc, but I would have loved using
>> something like this instead of jumping around carfax abbey (!) at
>> night.
> 
>     The core algorithm for the tool actually works pretty well for
> dealing with different RAID levels, as long as you know how much of
> each kind of data you're going to be using. (Although it's actually
> path-dependent -- write 100 GB of RAID-0 then 100 GB of RAID-1 can
> have different results than if you write them in the opposite order --
> but that's a kind of edge effect).
> 
>     Hugo.
> 

Re: btrfs raid assurance

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 2519 bytes --]

On Wed, Jul 26, 2017 at 08:12:19AM -0400, Austin S. Hemmelgarn wrote:
> On 2017-07-25 17:45, Hugo Mills wrote:
> >On Tue, Jul 25, 2017 at 11:29:13PM +0200, waxhead wrote:
> >>
> >>
> >>Hugo Mills wrote:
> >>>
> >>>>>    You can see about the disk usage in different scenarios with the
> >>>>>online tool at:
> >>>>>
> >>>>>http://carfax.org.uk/btrfs-usage/
> >>>>>
> >>>>>    Hugo.
> >>>>>
> >>As a side note, have you ever considered making this online tool
> >>(that should never go away just for the record) part of btrfs-progs
> >>e.g. a proper tool? I use it quite often (at least several timers
> >>per. month) and I would love for this to be a visual tool
> >>'btrfs-space-calculator' would be a great name for it I think.
> >>
> >>Imagine how nice it would be to run....
> >>
> >>btrfs-space-calculator -mraid1 -draid10 /dev/sda1 /dev/sdb1
> >>/dev/sdc2 /dev/sdd2 /dev/sde3 for example and instantly get
> >>something similar to my example below (no accuracy intended)
> >
> >    It's certainly a thought. I've already got the algorithm written
> >up. I'd have to resurrect my C skills, though, and it's a long way
> >down my list of things to do. :/
> >
> >    Also on the subject of this tool, I'd like to make it so that the
> >parameters get set in the URL, so that people can copy-paste the URL
> >of the settings they've got into IRC for discussion. However, that
> >would involve doing more JavaScript, which is possibly even lower down
> >my list of things to do than starting doing C again...

> Is the core logic posted somewhere?  Because if I have some time, I
> might write up a quick Python script to do this locally (it may not
> be as tightly integrated with the regular tools, but I can count on
> half a hand how many distros don't include Python by default).

   If it's going to be done in python, I might as well do it myself --
I can do python with my eyes closed. It's just C and JS I'm rusty with.

   There is a write-up of the usable-space algorithm somewhere. I
wrote it up in detail (with pseudocode) in a mail on this list. I've
also got several pages of LaTeX somewhere where I tried and failed to
prove the correctness of the formula. I'll see if I can dig them out
this evening.

   Hugo.

-- 
Hugo Mills             | How do you become King? You stand in the marketplace
hugo@... carfax.org.uk | and announce you're going to tax everyone. If you
http://carfax.org.uk/  | get out alive, you're King.
PGP: E2AB1DE4          |                                        Harry Harrison

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: btrfs raid assurance

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 2816 bytes --]

On Wed, Jul 26, 2017 at 12:27:20PM +0000, Hugo Mills wrote:
> On Wed, Jul 26, 2017 at 08:12:19AM -0400, Austin S. Hemmelgarn wrote:
> > On 2017-07-25 17:45, Hugo Mills wrote:
> > >On Tue, Jul 25, 2017 at 11:29:13PM +0200, waxhead wrote:
> > >>
> > >>
> > >>Hugo Mills wrote:
> > >>>
> > >>>>>    You can see about the disk usage in different scenarios with the
> > >>>>>online tool at:
> > >>>>>
> > >>>>>http://carfax.org.uk/btrfs-usage/
> > >>>>>
> > >>>>>    Hugo.
> > >>>>>
> > >>As a side note, have you ever considered making this online tool
> > >>(that should never go away just for the record) part of btrfs-progs
> > >>e.g. a proper tool? I use it quite often (at least several timers
> > >>per. month) and I would love for this to be a visual tool
> > >>'btrfs-space-calculator' would be a great name for it I think.
> > >>
> > >>Imagine how nice it would be to run....
> > >>
> > >>btrfs-space-calculator -mraid1 -draid10 /dev/sda1 /dev/sdb1
> > >>/dev/sdc2 /dev/sdd2 /dev/sde3 for example and instantly get
> > >>something similar to my example below (no accuracy intended)
> > >
> > >    It's certainly a thought. I've already got the algorithm written
> > >up. I'd have to resurrect my C skills, though, and it's a long way
> > >down my list of things to do. :/
> > >
> > >    Also on the subject of this tool, I'd like to make it so that the
> > >parameters get set in the URL, so that people can copy-paste the URL
> > >of the settings they've got into IRC for discussion. However, that
> > >would involve doing more JavaScript, which is possibly even lower down
> > >my list of things to do than starting doing C again...
> 
> > Is the core logic posted somewhere?  Because if I have some time, I
> > might write up a quick Python script to do this locally (it may not
> > be as tightly integrated with the regular tools, but I can count on
> > half a hand how many distros don't include Python by default).
> 
>    If it's going to be done in python, I might as well do it myself --
> I can do python with my eyes closed. It's just C and JS I'm rusty with.
> 
>    There is a write-up of the usable-space algorithm somewhere. I
> wrote it up in detail (with pseudocode) in a mail on this list. I've
> also got several pages of LaTeX somewhere where I tried and failed to
> prove the correctness of the formula. I'll see if I can dig them out
> this evening.

   Oh, and of course there's the JS from the website... that's not
minified, and should be readable (if not particularly well-commented).

   Hugo.

-- 
Hugo Mills             | How do you become King? You stand in the marketplace
hugo@... carfax.org.uk | and announce you're going to tax everyone. If you
http://carfax.org.uk/  | get out alive, you're King.
PGP: E2AB1DE4          |                                        Harry Harrison

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: btrfs raid assurance

[Austin S. Hemmelgarn]

On 2017-07-26 08:27, Hugo Mills wrote:
> On Wed, Jul 26, 2017 at 08:12:19AM -0400, Austin S. Hemmelgarn wrote:
>> On 2017-07-25 17:45, Hugo Mills wrote:
>>> On Tue, Jul 25, 2017 at 11:29:13PM +0200, waxhead wrote:
>>>>
>>>>
>>>> Hugo Mills wrote:
>>>>>
>>>>>>>     You can see about the disk usage in different scenarios with the
>>>>>>> online tool at:
>>>>>>>
>>>>>>> http://carfax.org.uk/btrfs-usage/
>>>>>>>
>>>>>>>     Hugo.
>>>>>>>
>>>> As a side note, have you ever considered making this online tool
>>>> (that should never go away just for the record) part of btrfs-progs
>>>> e.g. a proper tool? I use it quite often (at least several timers
>>>> per. month) and I would love for this to be a visual tool
>>>> 'btrfs-space-calculator' would be a great name for it I think.
>>>>
>>>> Imagine how nice it would be to run....
>>>>
>>>> btrfs-space-calculator -mraid1 -draid10 /dev/sda1 /dev/sdb1
>>>> /dev/sdc2 /dev/sdd2 /dev/sde3 for example and instantly get
>>>> something similar to my example below (no accuracy intended)
>>>
>>>     It's certainly a thought. I've already got the algorithm written
>>> up. I'd have to resurrect my C skills, though, and it's a long way
>>> down my list of things to do. :/
>>>
>>>     Also on the subject of this tool, I'd like to make it so that the
>>> parameters get set in the URL, so that people can copy-paste the URL
>>> of the settings they've got into IRC for discussion. However, that
>>> would involve doing more JavaScript, which is possibly even lower down
>>> my list of things to do than starting doing C again...
> 
>> Is the core logic posted somewhere?  Because if I have some time, I
>> might write up a quick Python script to do this locally (it may not
>> be as tightly integrated with the regular tools, but I can count on
>> half a hand how many distros don't include Python by default).
> 
>     If it's going to be done in python, I might as well do it myself --
> I can do python with my eyes closed. It's just C and JS I'm rusty with.
Same here ironically :)
> 
>     There is a write-up of the usable-space algorithm somewhere. I
> wrote it up in detail (with pseudocode) in a mail on this list. I've
> also got several pages of LaTeX somewhere where I tried and failed to
> prove the correctness of the formula. I'll see if I can dig them out
> this evening.
It looks like the Message-ID for the one on the mailing list is 
<20160311221703.GJ17196@carfax.org.uk>
I had forgotten that I'd archived that with the intent of actually doing 
something with it eventually...

Re: btrfs raid assurance

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 3384 bytes --]

On Wed, Jul 26, 2017 at 08:36:54AM -0400, Austin S. Hemmelgarn wrote:
> On 2017-07-26 08:27, Hugo Mills wrote:
> >On Wed, Jul 26, 2017 at 08:12:19AM -0400, Austin S. Hemmelgarn wrote:
> >>On 2017-07-25 17:45, Hugo Mills wrote:
> >>>On Tue, Jul 25, 2017 at 11:29:13PM +0200, waxhead wrote:
> >>>>
> >>>>
> >>>>Hugo Mills wrote:
> >>>>>
> >>>>>>>    You can see about the disk usage in different scenarios with the
> >>>>>>>online tool at:
> >>>>>>>
> >>>>>>>http://carfax.org.uk/btrfs-usage/
> >>>>>>>
> >>>>>>>    Hugo.
> >>>>>>>
> >>>>As a side note, have you ever considered making this online tool
> >>>>(that should never go away just for the record) part of btrfs-progs
> >>>>e.g. a proper tool? I use it quite often (at least several timers
> >>>>per. month) and I would love for this to be a visual tool
> >>>>'btrfs-space-calculator' would be a great name for it I think.
> >>>>
> >>>>Imagine how nice it would be to run....
> >>>>
> >>>>btrfs-space-calculator -mraid1 -draid10 /dev/sda1 /dev/sdb1
> >>>>/dev/sdc2 /dev/sdd2 /dev/sde3 for example and instantly get
> >>>>something similar to my example below (no accuracy intended)
> >>>
> >>>    It's certainly a thought. I've already got the algorithm written
> >>>up. I'd have to resurrect my C skills, though, and it's a long way
> >>>down my list of things to do. :/
> >>>
> >>>    Also on the subject of this tool, I'd like to make it so that the
> >>>parameters get set in the URL, so that people can copy-paste the URL
> >>>of the settings they've got into IRC for discussion. However, that
> >>>would involve doing more JavaScript, which is possibly even lower down
> >>>my list of things to do than starting doing C again...
> >
> >>Is the core logic posted somewhere?  Because if I have some time, I
> >>might write up a quick Python script to do this locally (it may not
> >>be as tightly integrated with the regular tools, but I can count on
> >>half a hand how many distros don't include Python by default).
> >
> >    If it's going to be done in python, I might as well do it myself --
> >I can do python with my eyes closed. It's just C and JS I'm rusty with.
> Same here ironically :)
> >
> >    There is a write-up of the usable-space algorithm somewhere. I
> >wrote it up in detail (with pseudocode) in a mail on this list. I've
> >also got several pages of LaTeX somewhere where I tried and failed to
> >prove the correctness of the formula. I'll see if I can dig them out
> >this evening.
> It looks like the Message-ID for the one on the mailing list is
> <20160311221703.GJ17196@carfax.org.uk>
> I had forgotten that I'd archived that with the intent of actually
> doing something with it eventually...

   Here's the write-up of my attempted proof of the optimality of the
current allocator algorithm:

http://carfax.org.uk/files/temp/btrfs-allocator-draft.pdf

   Section 1 is a general (allocator-agnostic) description of the
process. Section 2 finds a bound on how well _any_ allocator can
do. That's the formula (eq 9) used in the online btrfs-usage
tool. Section 3 described the current allocator. Section 4 is a failed
attempt at proving that the algorithm achieves the bound from section
2. I wasn't able to complete the proof.

   Hugo.

-- 
Hugo Mills             | Great films about cricket: Interview with the Umpire
hugo@... carfax.org.uk |
http://carfax.org.uk/  |
PGP: E2AB1DE4          |

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]