Question of stability

Question of stability

[Roy Sigurd Karlsbakk]

Hi all

I've been on this list for a year or so, and I have been following prog=
ress for some more. Are there any chances of btrfs stabilizing, as in t=
erms of usability in production? If so, how far are we from this?

Also, what about the RAID-[56] parts, they were announced more than a y=
ear ago, but still I can't see anything in the open.

Vennlige hilsener / Best regards

roy
--
Roy Sigurd Karlsbakk
(+47) 97542685
roy@karlsbakk.net
http://blogg.karlsbakk.net/
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt.=
 Det er et element=C3=A6rt imperativ for alle pedagoger =C3=A5 unng=C3=A5=
 eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste ti=
lfeller eksisterer adekvate og relevante synonymer p=C3=A5 norsk.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Question of stability

[Hendrik Fabelje]

On Sat, Sep 18, 2010 at 11:37 PM, Roy Sigurd Karlsbakk
<roy@karlsbakk.net> wrote:
> Hi all
>
> I've been on this list for a year or so, and I have been following progress for some more. Are there any chances of btrfs >stabilizing, as in terms of usability in production? If so, how far are we from this?
Hi,

I am using btrfs as my root filesystem on my Debian squeeze machine
for a few month now and so far I haven't experienced any problems.
It seems quite stable for me. I am not using raid functions, but am
also very interested in the progress in raid5/6.

Regards,
Hendrik

Re: Question of stability

[Roy Sigurd Karlsbakk]

----- Original Message -----
> On Sat, Sep 18, 2010 at 11:37 PM, Roy Sigurd Karlsbakk
> <roy@karlsbakk.net> wrote:
> > Hi all
> >
> > I've been on this list for a year or so, and I have been following
> > progress for some more. Are there any chances of btrfs >stabilizing=
,
> > as in terms of usability in production? If so, how far are we from
> > this?
> Hi,
>=20
> I am using btrfs as my root filesystem on my Debian squeeze machine
> for a few month now and so far I haven't experienced any problems.
> It seems quite stable for me. I am not using raid functions, but am
> also very interested in the progress in raid5/6.

I was more interested in large setups than a general install.

Question remains, when is btrfs supposed to be stable, as in usable for=
 large server setups?

Vennlige hilsener / Best regards

roy
--
Roy Sigurd Karlsbakk
(+47) 97542685
roy@karlsbakk.net
http://blogg.karlsbakk.net/
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt.=
 Det er et element=C3=A6rt imperativ for alle pedagoger =C3=A5 unng=C3=A5=
 eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste ti=
lfeller eksisterer adekvate og relevante synonymer p=C3=A5 norsk.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Question of stability

[C Anthony Risinger]

On Sep 18, 2010, at 6:55 PM, Roy Sigurd Karlsbakk <roy@karlsbakk.net>
wrote:

> ----- Original Message -----
>> On Sat, Sep 18, 2010 at 11:37 PM, Roy Sigurd Karlsbakk
>> <roy@karlsbakk.net> wrote:
>>> Hi all
>>>
>>> I've been on this list for a year or so, and I have been following
>>> progress for some more. Are there any chances of btrfs >stabilizing,
>>> as in terms of usability in production? If so, how far are we from
>>> this?
>> Hi,
>>
>> I am using btrfs as my root filesystem on my Debian squeeze machine
>> for a few month now and so far I haven't experienced any problems.
>> It seems quite stable for me. I am not using raid functions, but am
>> also very interested in the progress in raid5/6.
>
> I was more interested in large setups than a general install.
>
> Question remains, when is btrfs supposed to be stable, as in usable
> for large server setups?

Stable is a pretty subjective term; many don't even think ext4 is
stable.  I've used it on my personal machine since .30-31-ish without
problems, and on a server w/raid 1 for about a year (btrfs + lxc is
niiice, for VMs) also free of problems.

However, if you've been on the list you know that some do encounter
seemingly catastrophic problems, though the list is helpful in
recovering data.  So, it's really going to depends on your workload
and integrity needs.  I remeber someone recently using it for
continuous build servers successfully

Re: Question of stability

[Roy Sigurd Karlsbakk]

> Stable is a pretty subjective term; many don't even think ext4 is
> stable. I've used it on my personal machine since .30-31-ish without
> problems, and on a server w/raid 1 for about a year (btrfs + lxc is
> niiice, for VMs) also free of problems.
>=20
> However, if you've been on the list you know that some do encounter
> seemingly catastrophic problems, though the list is helpful in
> recovering data. So, it's really going to depends on your workload
> and integrity needs. I remeber someone recently using it for
> continuous build servers successfully

The term stable may be subjective at times, but for btrfs to be stable,=
 it needs a working filesystem, with offline or online fsck abilities, =
and allowing for what's in the idea of btrfs, that is, checksumming eve=
rything, allowing snapshots and rollbacks et cetera. If btrfs is only s=
table as in ext4, well, why not just use ext4? The whole reason for btr=
fs to exist is to bring something new into the Linux world, and if thos=
e features aren't stable, then btrfs isn't. It's as simple as that. Wou=
ld you buy a Subaru (or something) 4wd with a 2wd working?

Vennlige hilsener / Best regards

roy
--
Roy Sigurd Karlsbakk
(+47) 97542685
roy@karlsbakk.net
http://blogg.karlsbakk.net/
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt.=
 Det er et element=C3=A6rt imperativ for alle pedagoger =C3=A5 unng=C3=A5=
 eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste ti=
lfeller eksisterer adekvate og relevante synonymer p=C3=A5 norsk.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Question of stability

[C Anthony Risinger]

On Sat, Sep 18, 2010 at 9:00 PM, Roy Sigurd Karlsbakk <roy@karlsbakk.net> wrote:
>> Stable is a pretty subjective term; many don't even think ext4 is
>> stable. I've used it on my personal machine since .30-31-ish without
>> problems, and on a server w/raid 1 for about a year (btrfs + lxc is
>> niiice, for VMs) also free of problems.
>>
>> However, if you've been on the list you know that some do encounter
>> seemingly catastrophic problems, though the list is helpful in
>> recovering data. So, it's really going to depends on your workload
>> and integrity needs. I remeber someone recently using it for
>> continuous build servers successfully
>
> The term stable may be subjective at times, but for btrfs to be stable, it needs a working filesystem, with offline or online fsck abilities, and allowing for what's in the idea of btrfs, that is, checksumming everything, allowing snapshots and rollbacks et cetera. If btrfs is only stable as in ext4, well, why not just use ext4? The whole reason for btrfs to exist is to bring something new into the Linux world, and if those features aren't stable, then btrfs isn't. It's as simple as that. Would you buy a Subaru (or something) 4wd with a 2wd working?

whoa, relax; that's a terrible analogy ;-)

) the term stability is _always_ subjective
) fsck has nothing to do with the filesystem itself, and does not
contribute to it's operational stability
) checksums work fine
) snapshots work fine
) rollbacks are an implementation detail using snapshots; has nothing
to do w/filesystem, afiak
) ehm, i suppose you would use btrfs over ext4 because you need it's
features? beats me; you decide :-)
) ^^^^ have proper backup/failover options and it won't matter which you choose
) i'm sure that's not a reason ;-)
) ^^^^ btrfs has several pending/potential features/patches/branches
floating around such as raid5/6, hot data awareness, etc. -- these
unimplemented features (likely) do not detract from the stability of
what's implemented now

i apologize for the terseness, but i'm not sure what you're after
exactly -- you said you have been on the list for a year, and thus
should already have a pretty good idea of the current state, and what
you can/cannot do?  this (vague and _subjective_) question pops up
from time to time, along with questions about raid5/6, etc., and they
pretty much receive the same response i gave you:

) not everything possible/interesting is planned
) not everything planned is implemented
) some people run into big problems
) the majority likely does not
) use at your own risk
) many, including myself and the previous responder, are currently
using it for a wide range of capacities, successfully, and
collectively believe the minds responsible for btrfs must be rather
competent folk, because for the most part... things are pretty quiet
around here :-)

C Anthony

Re: Question of stability

[K. Richard Pixley]

  On 9/18/10 17:43 , C Anthony Risinger wrote:
> I remeber someone recently using it for
> continuous build servers successfully
That's probably me and I wouldn't call the effort "successful" yet.  
There are at least several outstanding problems including the limit on 
the number of links to a file.

More on all of these when I return from vacation next week.

--rich

Re: Question of stability

[C Anthony Risinger]

On Mon, Sep 20, 2010 at 10:12 AM, K. Richard Pixley <rich@noir.com> wro=
te:
> =A0On 9/18/10 17:43 , C Anthony Risinger wrote:
>>
>> I remeber someone recently using it for
>> continuous build servers successfully
>
> That's probably me and I wouldn't call the effort "successful" yet. =A0=
There
> are at least several outstanding problems including the limit on the =
number
> of links to a file.
>
> More on all of these when I return from vacation next week.

yes i think you're right; i actually was going to add that, but i
accidentally hit 'send' on my mobile right after typing 'successfully'
and never bothered to follow up with the last 2 or 3 sentences i had
planned :-), meh.

at any rate, while there is no doubt some issues (esp. with more
aggressive/non-typical setups), i still think there are many who are
using it successfully; at least this is the impression i get from my
statistically irrelevant sampling of the various forums, lists, etc. i
frequent.

C Anthony
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Question of stability

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 3028 bytes --]

On Sun, Sep 19, 2010 at 01:55:34AM +0200, Roy Sigurd Karlsbakk wrote:
> ----- Original Message -----
> > On Sat, Sep 18, 2010 at 11:37 PM, Roy Sigurd Karlsbakk
> > <roy@karlsbakk.net> wrote:
> > > Hi all
> > >
> > > I've been on this list for a year or so, and I have been following
> > > progress for some more. Are there any chances of btrfs >stabilizing,
> > > as in terms of usability in production? If so, how far are we from
> > > this?
> > Hi,
> > 
> > I am using btrfs as my root filesystem on my Debian squeeze machine
> > for a few month now and so far I haven't experienced any problems.
> > It seems quite stable for me. I am not using raid functions, but am
> > also very interested in the progress in raid5/6.
> 
> I was more interested in large setups than a general install.
> 
> Question remains, when is btrfs supposed to be stable, as in usable for large server setups?

   As has been pointed out by Anthony, there's no means of determining
when something is "stable" -- not just for filesystems, but for any
piece of software. All you can do is take a Bayesian approach: sum up
the number (and type) of failures, and compare it to the number of
user-hours that the software has been in use for, across all
installations. When that failure rate (and recovery rate) reaches the
point at which you're happy to use it in your situation -- whether
that's on your bleeding-edge desktop test box, or for running your
robotic heart surgeon -- you can call it stable. However, that point
has to be your decision for your particular use case.

   If you're now thinking, "but where do I get that information
from?", congratulations -- you now know nearly as much about the user
base as the btrfs developers. :) Your best bet is to keep an eye on
this mailing list, and take a look at the number and type of reported
failures. When that drops to the point that you feel safe, go ahead
and use it.

   An alternative approach is to install a btrfs set-up on your
internal development or test machines (you *do* have a test
infrastructure for your mission critical systems, right?), and hammer
it with the closest you can get to a real workload, and see what
happens. Again, this is a statistical approach. It's the best we've
got.

   At some point, we(*) hope, btrfs will have millions upon millions
of users, doing all kinds of bad things to it, and tiny fractions of
them will have problems. When that happens, someone will probably
start calling it stable, and the name will stick. Until then, many
people are happy with it for their uses, but nobody can (or will)
magically stick a label on a piece of code of this complexity and say
"it's stable now!"

   Hugo.

(*) Speaking as an interested nobody, rather than a developer.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
                  --- Be pure. Be vigilant. Behave. ---                  

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: Question of stability

[Chris Samuel]

On 19/09/10 07:37, Roy Sigurd Karlsbakk wrote:

> I've been on this list for a year or so, and I have been following
> progress for some more. Are there any chances of btrfs stabilizing,
> as in terms of usability in production? If so, how far are we from this?

I've been using btrfs in anger for a number of years (though not
using snapshots, subvolumes, etc) and am happy with it - though I
always make sure I've got plenty of free space.

However, I've been sufficiently worried about the checksum issues
being reported with newer kernels (still on 2.6.32 in Ubuntu 10.04)
that I'm considering deferring upgrading to 10.10 when it appears
to avoid the newer code.

I can see only one merge of patches to btrfs in the mainline kernel
since 2.6.35 was released on August 1st (merged August 10th), and
those came via the linux-2.6-block tree not from the btrfs devs so
I don't see any prospect of those issues being fixed in 2.6.36 either.

> Also, what about the RAID-[56] parts, they were announced more
> than a year ago, but still I can't see anything in the open.

Those are still out of tree I'm afraid.

cheers,
Chris
-- 
 Chris Samuel  :  http://www.csamuel.org/  :  Melbourne, VIC

Re: Question of stability

[Lubos Kolouch]

No, not stable!

Again, after powerloss, I have *two* damaged btrfs filesystems.

/home and also the backup volume, as the backup was running :(

Both show
parent transid verify failed on .... wanted ..... found ....

and the volumes cannot be mounted.

So until there is a way to somehow recover from this type of failures,
speaking about stability is a joke

Lubos Kolouch

Re: Question of stability

[Chris Mason]

On Mon, Sep 20, 2010 at 11:00:08AM +0000, Lubos Kolouch wrote:
> No, not stable!
> 
> Again, after powerloss, I have *two* damaged btrfs filesystems.

Please tell me more about your system.  I do extensive power fail
testing here without problems, and corruptions after powerloss are very
often caused by the actual hardware.

So, what kind of drives do you have, do they have writeback caching on,
and what are you layering on top of the drive between btrfs and the
kernel?

-chris

> 
> /home and also the backup volume, as the backup was running :(
> 
> Both show
> parent transid verify failed on .... wanted ..... found ....
> 
> and the volumes cannot be mounted.
> 
> So until there is a way to somehow recover from this type of failures,
> speaking about stability is a joke
> 
> Lubos Kolouch
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Question of stability

[Lubos Kolouch]

On Mon, 20 Sep 2010 07:30:57 -0400, Chris Mason wrote:

> On Mon, Sep 20, 2010 at 11:00:08AM +0000, Lubos Kolouch wrote:
>> No, not stable!
>> 
>> Again, after powerloss, I have *two* damaged btrfs filesystems.
> 
> Please tell me more about your system.  I do extensive power fail
> testing here without problems, and corruptions after powerloss are very
> often caused by the actual hardware.
> 
> So, what kind of drives do you have, do they have writeback caching on,
> and what are you layering on top of the drive between btrfs and the
> kernel?
> 
> -chris
> 

Hello Chris,

The system is running with 128GB SDD Samsung disk. The partition is 
encrypted with LUKS and on top btrfs filesystem is created.

The backup disk is a 160GB WD notebook disk connected via IDE->USB cable,
whole formatted with LUKS and btrfs on top.

Actually, if there was any way (non-standard) how to mount the system and 
recover even part of the data, I would be very grateful.

Lubos

Re: Question of stability

[Chris Mason]

On Mon, Sep 20, 2010 at 12:10:08PM +0000, Lubos Kolouch wrote:
> On Mon, 20 Sep 2010 07:30:57 -0400, Chris Mason wrote:
> 
> > On Mon, Sep 20, 2010 at 11:00:08AM +0000, Lubos Kolouch wrote:
> >> No, not stable!
> >> 
> >> Again, after powerloss, I have *two* damaged btrfs filesystems.
> > 
> > Please tell me more about your system.  I do extensive power fail
> > testing here without problems, and corruptions after powerloss are very
> > often caused by the actual hardware.
> > 
> > So, what kind of drives do you have, do they have writeback caching on,
> > and what are you layering on top of the drive between btrfs and the
> > kernel?
> > 
> > -chris
> > 
> 
> Hello Chris,
> 
> The system is running with 128GB SDD Samsung disk. The partition is 
> encrypted with LUKS and on top btrfs filesystem is created.

Ok, we're seeing consistent reports of corruptions after power failure
with dm-crypt.  The barriers must not be getting down to the device.

> 
> The backup disk is a 160GB WD notebook disk connected via IDE->USB cable,
> whole formatted with LUKS and btrfs on top.
> 
> Actually, if there was any way (non-standard) how to mount the system and 
> recover even part of the data, I would be very grateful.

We can definitely try.  Please send me email with the errors from
btrfsck and I'll work on a patch that gets you read only access.

Re: Question of stability

[Lubos Kolouch]

Chris Mason, Mon, 20 Sep 2010 08:13:07 -0400:

> On Mon, Sep 20, 2010 at 12:10:08PM +0000, Lubos Kolouch wrote:
>> On Mon, 20 Sep 2010 07:30:57 -0400, Chris Mason wrote:
>> 
>> > On Mon, Sep 20, 2010 at 11:00:08AM +0000, Lubos Kolouch wrote:
>> >> No, not stable!
>> >> 
>> >> Again, after powerloss, I have *two* damaged btrfs filesystems.
>> > 
>> > Please tell me more about your system.  I do extensive power fail
>> > testing here without problems, and corruptions after powerloss are
>> > very often caused by the actual hardware.
>> > 
>> > So, what kind of drives do you have, do they have writeback caching
>> > on, and what are you layering on top of the drive between btrfs and
>> > the kernel?
>> > 
>> > -chris
>> > 
>> > 
>> Hello Chris,
>> 
>> The system is running with 128GB SDD Samsung disk. The partition is
>> encrypted with LUKS and on top btrfs filesystem is created.
> 
> Ok, we're seeing consistent reports of corruptions after power failure
> with dm-crypt.  The barriers must not be getting down to the device.
> 
> 
>> The backup disk is a 160GB WD notebook disk connected via IDE->USB
>> cable, whole formatted with LUKS and btrfs on top.
>> 
>> Actually, if there was any way (non-standard) how to mount the system
>> and recover even part of the data, I would be very grateful.
> 
> We can definitely try.  Please send me email with the errors from
> btrfsck and I'll work on a patch that gets you read only access.

Just for the record - thanks to Chris I could recover the data in read-
only mode, thanks again.

So are there any recommendations (mount options etc.) for running btrfs
on LUKS encrypted partition?

Thank you 

Lubos

Re: Question of stability

[Chris Mason]

On Wed, Sep 22, 2010 at 02:04:57PM +0000, Lubos Kolouch wrote:
> Chris Mason, Mon, 20 Sep 2010 08:13:07 -0400:
> 
> > On Mon, Sep 20, 2010 at 12:10:08PM +0000, Lubos Kolouch wrote:
> >> On Mon, 20 Sep 2010 07:30:57 -0400, Chris Mason wrote:
> >> 
> >> > On Mon, Sep 20, 2010 at 11:00:08AM +0000, Lubos Kolouch wrote:
> >> >> No, not stable!
> >> >> 
> >> >> Again, after powerloss, I have *two* damaged btrfs filesystems.
> >> > 
> >> > Please tell me more about your system.  I do extensive power fail
> >> > testing here without problems, and corruptions after powerloss are
> >> > very often caused by the actual hardware.
> >> > 
> >> > So, what kind of drives do you have, do they have writeback caching
> >> > on, and what are you layering on top of the drive between btrfs and
> >> > the kernel?
> >> > 
> >> > -chris
> >> > 
> >> > 
> >> Hello Chris,
> >> 
> >> The system is running with 128GB SDD Samsung disk. The partition is
> >> encrypted with LUKS and on top btrfs filesystem is created.
> > 
> > Ok, we're seeing consistent reports of corruptions after power failure
> > with dm-crypt.  The barriers must not be getting down to the device.
> > 
> > 
> >> The backup disk is a 160GB WD notebook disk connected via IDE->USB
> >> cable, whole formatted with LUKS and btrfs on top.
> >> 
> >> Actually, if there was any way (non-standard) how to mount the system
> >> and recover even part of the data, I would be very grateful.
> > 
> > We can definitely try.  Please send me email with the errors from
> > btrfsck and I'll work on a patch that gets you read only access.
> 
> Just for the record - thanks to Chris I could recover the data in read-
> only mode, thanks again.
> 
> So are there any recommendations (mount options etc.) for running btrfs
> on LUKS encrypted partition?

I would suggest turning off the drives writeback cache.  hdparm -W 0, it
must be run after every boot.

-chris

Re: Question of stability

[Stephan von Krawczynski]

On Mon, 20 Sep 2010 07:30:57 -0400
Chris Mason <chris.mason@oracle.com> wrote:

> On Mon, Sep 20, 2010 at 11:00:08AM +0000, Lubos Kolouch wrote:
> > No, not stable!
> > 
> > Again, after powerloss, I have *two* damaged btrfs filesystems.
> 
> Please tell me more about your system.  I do extensive power fail
> testing here without problems, and corruptions after powerloss are very
> often caused by the actual hardware.
> 
> So, what kind of drives do you have, do they have writeback caching on,
> and what are you layering on top of the drive between btrfs and the
> kernel?
> 
> -chris

Chris, the actual way how a fs was damaged must not be relevant. From a new fs
design one should expect the tree can be mounted no matter what corruption took
place up to the case where the fs is indeed empty after mounting because it
was completely corrupted. If parts were corrupt then the fs should either be
able to assist the user in correcting the damages _online_ or at least simply
exclude the damaged fs parts from the actual mounted fs tree. The basic
thought must be "show me what you have" and not "shit, how do I get access to
the working but not mountable fs parts again?".
Would you buy a car that refuses to drive if the ash tray is broken?

-- 
Regards,
Stephan

Re: Question of stability

[Chris Mason]

On Mon, Sep 20, 2010 at 02:21:15PM +0200, Stephan von Krawczynski wrote:
> On Mon, 20 Sep 2010 07:30:57 -0400
> Chris Mason <chris.mason@oracle.com> wrote:
> 
> > On Mon, Sep 20, 2010 at 11:00:08AM +0000, Lubos Kolouch wrote:
> > > No, not stable!
> > > 
> > > Again, after powerloss, I have *two* damaged btrfs filesystems.
> > 
> > Please tell me more about your system.  I do extensive power fail
> > testing here without problems, and corruptions after powerloss are very
> > often caused by the actual hardware.
> > 
> > So, what kind of drives do you have, do they have writeback caching on,
> > and what are you layering on top of the drive between btrfs and the
> > kernel?
> > 
> > -chris
> 
> Chris, the actual way how a fs was damaged must not be relevant. From a new fs
> design one should expect the tree can be mounted no matter what corruption took
> place up to the case where the fs is indeed empty after mounting because it
> was completely corrupted. If parts were corrupt then the fs should either be
> able to assist the user in correcting the damages _online_ or at least simply
> exclude the damaged fs parts from the actual mounted fs tree. The basic
> thought must be "show me what you have" and not "shit, how do I get access to
> the working but not mountable fs parts again?".
> Would you buy a car that refuses to drive if the ash tray is broken?

Definitely, this has always been one of our goals.  Step one is a better
btrfsck, which is in progress now.

Step two is being able to do more of this online.

-chris