Re: 40TB volume taking over 16 hours to mount, any ideas?

[Duncan <1i5t5.duncan@cox.net>]

Jose Ildefonso Camargo Tolosa posted on Sat, 09 Aug 2014 22:58:37 -0500 as
excerpted:

> Do you think I will have better luck with 3.16? or maybe it is that this
> filesystem has so many errors (remember the btrfs check output) that it
> will take a really long time to mount because it is trying to correct
> this?

As a user I'd give up on that mount.

There are two critical patches in the pipeline ATM, that should hopefully 
hit 3.17-rc1 next weekend.  The one's already posted (See the Fix csum 
tree corruption patch first posted just under 12 hours ago as I type 
this), but that's a longer term fix.  The other was traced down late last 
week but I don't believe a proper patch has been posted yet.  That's the 
one you likely need here.  Of course you could cherrypick it when posted.

Tho either way I think it's likely that the filesystem is toast and 
you'll end up doing a mkfs on it, hopefully with those patches helping to 
prevent a repeat.

What I'd try at this point is btrfs restore, tho you'll need somewhere 
else to put the restored data, and you'll have to redo file ownership and 
permissions as that's not restored, only the data files.

Or restore from backup, your choice, but you said it was remote and 
you're looking at over a week's worth of downloading.

Either way, the question then comes up of what to use when you do a new 
mkfs.  My personal feeling?  Btrfs isn't yet fully stable, and there's a 
very real possibility that one may have to restore from backup, so one 
should be prepared for that.  Given the size of the data store you're 
working with and the remote nature of that backup, with access over 
limited-speed pipes, I wonder if btrfs is really an appropriate choice 
for you at this point.  If you believe the features of btrfs and the 
chance to work with something so leading/bleeding edge are worth the 
current pain and are prepared to redo that restore again should it be 
necessary, then yes, btrfs is a good choice.  OTOH, if you want something 
that reliably "just works" at this point, consider a more mature 
filesystem.  It may not have btrfs' bells and whistles, but a boring 
"just works, reliably", might be what you want.

I guess xfs is the standard recommendation for big-data sizes and it is 
said to be long past the "better have a UPS" days, or of course the 
default ext4.  Personally I've had real good luck with reiserfs (since 
data=ordered by default at least, the early data=writeback days were 
where it got its bad rep), but it's better adapted to smaller files, 
while I'd guess with 40 TB, your files are likely big as well.

You can of course try btrfs again in a year or so, when it should have 
matured quite a bit.  I actually did that after my first try at btrfs, 
leaving for a time then coming back, and was impressed at how much it had 
matured in the mean time.  Additionally, my use-case was different as the 
first time I tried it I was still on spinning rust; now all my btrfs are 
on SSD, and I still use reiserfs for my spinning rust -- tho I've nowhere 
near the double-digit TB scale you're doing.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman