Re: Status of FST and mount times

["Austin S. Hemmelgarn" <ahferroin7@gmail.com>]

On 2018-02-15 11:58, Ellis H. Wilson III wrote:
> On 02/15/2018 11:51 AM, Austin S. Hemmelgarn wrote:
>> There are scaling performance issues with directory listings on BTRFS 
>> for directories with more than a few thousand files, but they're not 
>> well documented (most people don't hit them because most applications 
>> are designed around the expectation that directory listings will be 
>> slow in big directories), and I would not expect them to be much of an 
>> issue unless you're dealing with tens of thousands of files and 
>> particularly slow storage.
> 
> Understood -- thanks.  Then plan is to keep it to around 1k entries per 
> directory.  We've done some fairly concrete testing here to find the 
> fall-off point for dirent caching in BTRFS, and the sweet-spot between 
> having a large number of small directories cached vs. a few massive 
> directories cached.  ~1k seems most palatable for our use-case and 
> directory tree structure.
Yeah, in my own experience this starts to get noticeable on slower 
storage around about 4k or more entries in a directory, but it ends up 
depending on the hardware to a certain extent and the rest of the system 
as well (something Samba does seems to make it significantly worse than 
listing locally for example, while NFS seems to be only be worse because 
of network latency).
> 
>> I've only ever lost a BTRFS volume to a power failure _once_ in the 
>> multiple years I've been using it, and that ended up being because the 
>> power failure trashed the storage device pretty severely (it was 
>> super-cheap flash storage).  I do know however that there are people 
>> who have had much worse results than me.
> 
> Good to know.  We'll be running power-fail testing over the next couple 
> months.  I'm waiting for some hardware to arrive presently.  We'll 
> power-cycle fairly large filesystems a few thousand times before we deem 
> it safe to ship.  If there are latent bugs in BTRFS still w.r.t. 
> power-fail, I can guarantee we'll trip over them...
Most of my own experience regarding power failures with BTRFS is on 
SSD's.  We actually use it on the embedded systems we build where I 
work, and a lot of our customers don't have the most reliable mains 
power (or they're too lazy to shut off the computer properly before 
flipping the main breaker for the machine to power it off for the 
evening), so some of our systems may see power failures on an almost 
daily basis.  Despite that, we've never had issues with BTRFS not 
recovering by itself, though we do have a very read-heavy workload with 
very infrequent writes, so that may be part of why it's worked so well 
for us.