finobt option for end user

finobt option for end user

[Alphazo]

[-- Attachment #1.1: Type: text/plain, Size: 754 bytes --]

Hello,

I'm pretty new to XFS. I'm considering moving away from ext4 to XFS because
of the new self-describing option, performance and reliability improvements
that XFS went through over the past year. Now I'm puzzled with the new free
inode btree option (finobt). I tried to find some documentation about it
but couldn't find the pros or cons. So from an end-user perspective with a
couple of TB worth of photos:
- Does it improve overall reliability?
- Does it provide faster fsck/repair?
- Does it improve any read or write operation?
- Is it safe to use and does it recover as well as with finobt=0?
- What is the typical case for enabling it and would you recommend using it
for any new fs creation?

Thank you in advance for your pointers.
Alphazo

[-- Attachment #1.2: Type: text/html, Size: 900 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: finobt option for end user

[Michael L. Semon]

On 12/20/14 17:52, Alphazo wrote:
> Hello,
> 
> I'm pretty new to XFS. I'm considering moving away from ext4 to XFS because
> of the new self-describing option, performance and reliability improvements
> that XFS went through over the past year. Now I'm puzzled with the new free
> inode btree option (finobt). I tried to find some documentation about it
> but couldn't find the pros or cons. So from an end-user perspective with a
> couple of TB worth of photos:
> - Does it improve overall reliability?
> - Does it provide faster fsck/repair?
> - Does it improve any read or write operation?
> - Is it safe to use and does it recover as well as with finobt=0?
> - What is the typical case for enabling it and would you recommend using it
> for any new fs creation?
> 
> Thank you in advance for your pointers.
> Alphazo

As a user...

You could try it out and see:

mkfs.xfs -m crc=1,finobt=1 <block_device_to_format>

Here, on old junk hardware, finobt aids greatly in multitasking on 
small-file creation.  It may not have the same effect on your hardware.

finobt has given very, very little trouble since it was accepted into the 
kernel code and xfsprogs.

Recovery and reliability have become roughly equal between old XFS, new 
XFS, and new XFS with finobt.  They just approach the problem in different 
ways.  With old XFS, I rely heavily on xfs_repair and xfsdump to show 
filesystem issues.  With new XFS, the kernel code is more likely to complain 
as the issues are happening.

finobt introduces the slightest bit of overhead that might be noticed at the 
point of resource exhaustion.  Otherwise, its overhead might be hard to spot.

As for safety and reliability, I'm working with kernel 3.18 with the drives' 
write caches shut off and using the latest xfsprogs.  It works great.  The 
new v5-superblock XFS progresses at a fast pace, and it helps greatly to 
keep up with it.  kernel 3.18 seems to be a better kernel, period.  YMMV.

Try a test filesystem first and test to your liking first.  If something 
goes wrong, you should find that out in two weeks of testing the file 
system.

Just curious, though, with that volume of images, is there a possibility of 
making a read-only filesystem?  That would take a lot of the safety and 
recovery issues out of the equation, even if you stick with ext4.  It would 
make your main concern to sha256sum the files so that you can check them 
later for bit-rot.

Good luck!

Michael

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: finobt option for end user

[Alphazo]

[-- Attachment #1.1: Type: text/plain, Size: 3427 bytes --]

Thanks for the detailed answer.
Regarding your last comment on mounting my volume read-only it might not be
possible since I update the metadatas of my pictures quite often and thus
even on older pictures. Part of my backup strategy, besides my unRAID NAS,
that BTW has recently officially switched from ReiserFS to XFS, is a
dedicated drive storing backup created by "bup" with the optional PAR2
checksum activated so I get incrementa and de-duplicated backup as well as
protection against bit-rot.

PS: I was so excited about the new XFS metadata checksum that I went ahead
and added a section about it to the ArchLinux official wiki. Now I'm trying
to figure out if the finobt bit should be added as well under the
performance section.
https://wiki.archlinux.org/index.php/XFS#Integrity


On Sun, Dec 21, 2014 at 9:38 PM, Michael L. Semon <mlsemon35@gmail.com>
wrote:

> On 12/20/14 17:52, Alphazo wrote:
> > Hello,
> >
> > I'm pretty new to XFS. I'm considering moving away from ext4 to XFS
> because
> > of the new self-describing option, performance and reliability
> improvements
> > that XFS went through over the past year. Now I'm puzzled with the new
> free
> > inode btree option (finobt). I tried to find some documentation about it
> > but couldn't find the pros or cons. So from an end-user perspective with
> a
> > couple of TB worth of photos:
> > - Does it improve overall reliability?
> > - Does it provide faster fsck/repair?
> > - Does it improve any read or write operation?
> > - Is it safe to use and does it recover as well as with finobt=0?
> > - What is the typical case for enabling it and would you recommend using
> it
> > for any new fs creation?
> >
> > Thank you in advance for your pointers.
> > Alphazo
>
> As a user...
>
> You could try it out and see:
>
> mkfs.xfs -m crc=1,finobt=1 <block_device_to_format>
>
> Here, on old junk hardware, finobt aids greatly in multitasking on
> small-file creation.  It may not have the same effect on your hardware.
>
> finobt has given very, very little trouble since it was accepted into the
> kernel code and xfsprogs.
>
> Recovery and reliability have become roughly equal between old XFS, new
> XFS, and new XFS with finobt.  They just approach the problem in different
> ways.  With old XFS, I rely heavily on xfs_repair and xfsdump to show
> filesystem issues.  With new XFS, the kernel code is more likely to
> complain
> as the issues are happening.
>
> finobt introduces the slightest bit of overhead that might be noticed at
> the
> point of resource exhaustion.  Otherwise, its overhead might be hard to
> spot.
>
> As for safety and reliability, I'm working with kernel 3.18 with the
> drives'
> write caches shut off and using the latest xfsprogs.  It works great.  The
> new v5-superblock XFS progresses at a fast pace, and it helps greatly to
> keep up with it.  kernel 3.18 seems to be a better kernel, period.  YMMV.
>
> Try a test filesystem first and test to your liking first.  If something
> goes wrong, you should find that out in two weeks of testing the file
> system.
>
> Just curious, though, with that volume of images, is there a possibility of
> making a read-only filesystem?  That would take a lot of the safety and
> recovery issues out of the equation, even if you stick with ext4.  It would
> make your main concern to sha256sum the files so that you can check them
> later for bit-rot.
>
> Good luck!
>
> Michael
>
>

[-- Attachment #1.2: Type: text/html, Size: 4286 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: finobt option for end user

[Dave Chinner]

On Sat, Dec 20, 2014 at 11:52:57PM +0100, Alphazo wrote:
> Hello,
> 
> I'm pretty new to XFS. I'm considering moving away from ext4 to XFS because
> of the new self-describing option, performance and reliability improvements
> that XFS went through over the past year. Now I'm puzzled with the new free
> inode btree option (finobt). I tried to find some documentation about it
> but couldn't find the pros or cons. So from an end-user perspective with a
> couple of TB worth of photos:
> - Does it improve overall reliability?

No effect on reliability, good or bad.

> - Does it provide faster fsck/repair?

No, if anything, it slows it down because there's more metadata to
verify and rebuild.

> - Does it improve any read or write operation?

No.

> - Is it safe to use and does it recover as well as with finobt=0?

Yes and yes.

> - What is the typical case for enabling it

It improves performance on aged filesystems i.e. months or years
down the track when you've added and removed millions of
files to/from the filesystem.

> and would you recommend using it
> for any new fs creation?

Consider the fact that "-m crc=1,finobt=1" will be made the mkfs
default in 3-6 months time - that will probably co-incide with a
xfsprogs 3.3 release....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: finobt option for end user

[Alphazo]

[-- Attachment #1.1: Type: text/plain, Size: 1806 bytes --]

Thanks Dave and Michael for the very informative answers.

I have updated the official ArchLinux XFS wiki page based upon the
information I gathered here so it can benefit to others.

https://wiki.archlinux.org/index.php/XFS#Free_inode_btree

Please do not hesitate to let me know if you see any mistake in my overview.

Cheers,
Alphazo

On Sun, Dec 21, 2014 at 11:18 PM, Dave Chinner <david@fromorbit.com> wrote:

> On Sat, Dec 20, 2014 at 11:52:57PM +0100, Alphazo wrote:
> > Hello,
> >
> > I'm pretty new to XFS. I'm considering moving away from ext4 to XFS
> because
> > of the new self-describing option, performance and reliability
> improvements
> > that XFS went through over the past year. Now I'm puzzled with the new
> free
> > inode btree option (finobt). I tried to find some documentation about it
> > but couldn't find the pros or cons. So from an end-user perspective with
> a
> > couple of TB worth of photos:
> > - Does it improve overall reliability?
>
> No effect on reliability, good or bad.
>
> > - Does it provide faster fsck/repair?
>
> No, if anything, it slows it down because there's more metadata to
> verify and rebuild.
>
> > - Does it improve any read or write operation?
>
> No.
>
> > - Is it safe to use and does it recover as well as with finobt=0?
>
> Yes and yes.
>
> > - What is the typical case for enabling it
>
> It improves performance on aged filesystems i.e. months or years
> down the track when you've added and removed millions of
> files to/from the filesystem.
>
> > and would you recommend using it
> > for any new fs creation?
>
> Consider the fact that "-m crc=1,finobt=1" will be made the mkfs
> default in 3-6 months time - that will probably co-incide with a
> xfsprogs 3.3 release....
>
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david@fromorbit.com
>

[-- Attachment #1.2: Type: text/html, Size: 2729 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs