Re: Extremely slow metadata performance

[Duncan <1i5t5.duncan@cox.net>]

John Goerzen posted on Thu, 05 Dec 2013 11:52:04 -0600 as excerpted:

> Hello,
> 
> I have observed extremely slow metadata performance with btrfs. This may
> be a bit of a nightmare scenario; it involves untarring a backup of
> 1.6TB of backuppc data, which contains millions of hardlinks and much
> data, onto USB 2.0 disks.

> Is this behavior known and expected?

Yes.  Btrfs doesn't do well with lots of hardlinks and indeed until 
relatively recently had a hard-limit on the number of hardlinks possible 
within a directory, that hardlink-heavy use-cases would regularly hit.  
That was worked around, but there's an additional level of indirection 
once the first level link-pool is filled, and you're not the first to 
have observed that btrfs performance isn't the best in that sort of 
scenario.  That's known.

Other filesystems will probably do quite a bit better for hardlink style 
backups and other hardlink-heavy use-cases.  Either that, or consider 
using btrfs, but with some other form of backup, possibly btrfs 
snapshots, or COW reflinks.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman