Re: Question about logbsize default value

[Gionatan Danti <g.danti@assyoma.it>]

Il 26-10-2019 01:39 Dave Chinner ha scritto:
> Again, it's a trade-off.
> 
> 256kB iclogs mean that a crash can leave an unrecoverable 2MB hole
> in the journal, while 32kB iclogs means it's only 256kB.

Sure, but a crash will always cause the loss of unsynced data, 
especially when using deferred logging and/or deferred allocation, 
right?

> 256kB iclogs mean 2MB of memory usage per filesystem, 32kB is only
> 256kB. We have users with hundreds of individual XFS filesystems
> mounted on single machines, and so 256kB iclogs is a lot of wasted
> memory...

Just wondering: 1000 filesystems with 256k logbsize would result in 2 GB 
of memory consumed by journal buffers. Is this considered too much 
memory for a system managing 1000 filesystems? The pagecache write back 
memory consumption on these systems (probably equipped with 10s GB of 
RAM) would dwarfs any journal buffers, no?

> On small logs and filesystems, 256kB iclogs doesn't provide any real
> benefit because throughput is limited by log tail pushing (metadata
> writeback), not async transaction throughput.
> 
> It's not uncommon for modern disks to have best throughput and/or
> lowest latency at IO sizes of 128kB or smaller.
> 
> If you have lots of NVRAM in front of your spinning disks, then log
> IO sizes mostly don't matter - they end up bandwidth limited before
> the iclog size is an issue.

Yes, this matches my observation.

> Testing on a pristine filesystem doesn't show what happens as the
> filesystem ages over years of constant use, and so what provides
> "best performance on empty filesystem" often doesn't provide best
> long term production performance.
> 
> And so on.
> 
> Storage is complex, filesystems are complex, and no one setting is
> right for everyone. The defaults are intended to be "good enough" in
> the majority of typical user configs.

Yep.

> 
> For you're specific storage setup, yes.
> 
>> If you, do you suggest to always set logbsize
>> to the maximum supported value?
> 
> No. I recommend that people use the defaults, and only if there are
> performance issues with their -actual production workload- should
> they consider changing anything.
> 
> Benchmarks rarely match the behaviour of production workloads -
> tuning for benchmarks can actively harm production performance,
> especially over the long term...
> 
> Cheers,
> 
> Dave.

Ok, very clear.
Thank you so much.

-- 
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8