A little confused about what remains to make a stable release

A little confused about what remains to make a stable release

[Daniel Farina]

Hello List,

I have been tracking the development of btrfs for some time, as the
built-in support for snapshotting would be of great convenience for
relational database use cases. I have been crawling the wiki
(especially the FAQ), but I still don't have a clear sense of "what's
left" *besides* the need for a 'fsck' utility that can be called
absolutely vital.

That, and testing and bug reports.

>From the materials I've been able to find, it's hard for me to get a
sense of how one could assist the project towards being recommended
for general use; do the denizens of this list has a sense of what
those things might be? (Or a link?)

--
fdr

Re: A little confused about what remains to make a stable release

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 1860 bytes --]

On Wed, Nov 17, 2010 at 02:27:39PM -0800, Daniel Farina wrote:
> I have been tracking the development of btrfs for some time, as the
> built-in support for snapshotting would be of great convenience for
> relational database use cases. I have been crawling the wiki
> (especially the FAQ), but I still don't have a clear sense of "what's
> left" *besides* the need for a 'fsck' utility that can be called
> absolutely vital.
> 
> That, and testing and bug reports.

   This question (and answer) should probably go in the FAQ...

   Nobody is going to magically stick a label on btrfs and say "it's
stable now!" Software -- particularly something as complex as this --
just doesn't work that way.

   It's stable *for you* when it functions with the workloads *you*
expect of it, with a failure rate that is acceptable *to you*.

   For what I'm using it for right now, it's already stable by that
definition, *for me*.

> From the materials I've been able to find, it's hard for me to get a
> sense of how one could assist the project towards being recommended
> for general use; do the denizens of this list has a sense of what
> those things might be? (Or a link?)

   The primary things you can do: Use it, test it, file bug reports.
Do this with, as close as you can, the use-cases (IOPs/s, feature
uses, data sizes) that you want to use it for.

   Beyond that: Fix bugs. Add the features that you think are
important. Add features that other people think are important (see the
"Project Ideas" page on the wiki for the latter).

   Hugo.

</2-penn'orth>

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
  --- The trouble with you, Ibid, is you think you know everything. ---  
                                                                         

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: A little confused about what remains to make a stable release

[Anthony Roberts]

 Hello,

>    It's stable *for you* when it functions with the workloads *you*
> expect of it, with a failure rate that is acceptable *to you*.

 I think there's a few ancillary things like a working fsck needed
 before it can even be recommended for widespread use, even to users
 willing to risk any residual bugs. IIRC at this point the utilities
 don't even aspire to provide basic recovery functionality (though
 Chris has posted that fsck is coming).

 Beyond that, the management capabilities at this point don't look
 ready for long term use in a production environment. By this I
 mean adding/removing disks, reshaping arrays, etc. Without that I
 might use BTRFS on top of LVM/RAID just like any other filesystem,
 and there's features I'm looking forward to even if I that's all
 I can do, but without robust management features there's certain
 environments where it just doesn't make sense yet.

 There's one or two other things I'm keeping an eye on. That
 limitation on the number of hardlinks you can have in a directory
 is kinda irksome. Also, dedup needs a way to verify/dedup safely
 before people can start doing stuff like deduping live VM images.

 -Anthony

Re: A little confused about what remains to make a stable release

[Daniel Farina]

On Wed, Nov 17, 2010 at 4:46 PM, Anthony Roberts
<btrfs-devel@arbitraryconstant.com> wrote:
> I think there's a few ancillary things like a working fsck needed
> before it can even be recommended for widespread use, even to users
> willing to risk any residual bugs. IIRC at this point the utilities
> don't even aspire to provide basic recovery functionality (though
> Chris has posted that fsck is coming).

Yes, I think this is a rather big one, and one of the more directly
addressed issues given the documentation at hand. I'm glad it looks
like it is getting attention.

> Beyond that, the management capabilities at this point don't look
> ready for long term use in a production environment. By this I
> mean adding/removing disks, reshaping arrays, etc. Without that I
> might use BTRFS on top of LVM/RAID just like any other filesystem,
> and there's features I'm looking forward to even if I that's all
> I can do, but without robust management features there's certain
> environments where it just doesn't make sense yet.

I'd really like to see btrfs become able to simplify my life by
removing LVM and/or RAID from the equation, that would be a very
compelling reason to use it for me. Is there a place with a little
more detail on what's missing to complete the storage-pool
functionality?

> There's one or two other things I'm keeping an eye on. That
> limitation on the number of hardlinks you can have in a directory
> is kinda irksome.

I've heard this may require a format change, which if correct could be
the cause of some reticence to use btrfs for very large file systems.

> Also, dedup needs a way to verify/dedup safely
> before people can start doing stuff like deduping live VM images.

Hmm. Is that going to require a format change? Somehow it seems less likely...

Cheers.

--
fdr

Re: A little confused about what remains to make a stable release

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 1967 bytes --]

On Wed, Nov 17, 2010 at 05:46:30PM -0700, Anthony Roberts wrote:
> >   It's stable *for you* when it functions with the workloads *you*
> >expect of it, with a failure rate that is acceptable *to you*.
> 
> I think there's a few ancillary things like a working fsck needed
> before it can even be recommended for widespread use, even to users
> willing to risk any residual bugs. IIRC at this point the utilities
> don't even aspire to provide basic recovery functionality (though
> Chris has posted that fsck is coming).
> 
> Beyond that, the management capabilities at this point don't look
> ready for long term use in a production environment. By this I
> mean adding/removing disks,

   That much is already there and working.

> reshaping arrays, etc. Without that I
> might use BTRFS on top of LVM/RAID just like any other filesystem,
> and there's features I'm looking forward to even if I that's all
> I can do, but without robust management features there's certain
> environments where it just doesn't make sense yet.

   What do you think is missing? Could you create and maintain a
wishlist page on the wiki[1], and populate it with all the things that
people need for production use? (This is an ongoing task -- track
what's actually finished and remove it; track what's currently being
worked on and mark it as such; keep an eye on discussions on the
mailing list for things that people need...)

> There's one or two other things I'm keeping an eye on. That
> limitation on the number of hardlinks you can have in a directory
> is kinda irksome. Also, dedup needs a way to verify/dedup safely
> before people can start doing stuff like deduping live VM images.

   Hugo.

[1] https://btrfs.wiki.kernel.org/index.php/Main_Page

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
       --- Someone's been throwing dead sheep down my Fun Well ---       

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: A little confused about what remains to make a stable release

[Anthony Roberts]

>> Beyond that, the management capabilities at this point don't look
>> ready for long term use in a production environment. By this I
>> mean adding/removing disks,
>
>    That much is already there and working.

 Only for the basics though, yes? Disks can be added, but IIRC you can't 
 really control RAID levels for new disks. To do something like start out 
 with a RAID1 mirror and add a 4 drive RAID5 with disks of different size 
 to it, you'll have to be doing it on top of another RAID layer.

 BTRFS hopes to replace MD and LVM, but for that to happen it will 
 ultimately be necessary to be able to manage sets of disks just like you 
 can with separate MD devices, or hardware RAID, or SANs.

 I mean, there might be a few cases where you don't have quite as much 
 granularity because it gets weird when you cross layers like this, but 
 broadly speaking btrfs I think it makes sense for a volume manager to 
 know about and be able to manage sets of disks.

>    What do you think is missing? Could you create and maintain a
> wishlist page on the wiki[1], and populate it with all the things 
> that
> people need for production use?

 I'd be happy to contribute my thoughts, but isn't there already a 
 project ideas page? Actually I think much of what I've mentioned is 
 already covered under the "changing raid levels" entry.

 For sets of disks, I'd be happy to add an entry, but before I take it 
 upon myself to edit the wiki it probably makes sense to give Chris or 
 someone an opportunity to disagree that it's a good idea in the first 
 place.

 Regards,

 -Anthony