Re: Question about logbsize default value

Gionatan Danti
2019-10-26T09:54:09Z

Il 26-10-2019 01:39 Dave Chinner ha scritto:
> Again, it's a trade-off.
>
> 256kB iclogs mean that a crash can leave an unrecoverable 2MB hole
> in the journal, while 32kB iclogs means it's only 256kB.

Sure, but a crash will always cause the loss of unsynced data,
especially when using deferred logging and/or deferred allocation,
right?

> 256kB iclogs mean 2MB of memory usage per filesystem, 32kB is only
> 256kB. We have users with hundreds of individual XFS filesystems
> mounted on single machines, and so 256kB iclogs is a lot of wasted
> memory...

Just wondering: 1000 filesystems with 256k logbsize would result in 2 GB
of memory consumed by journal buffers. Is this considered too much
memory for a system managing 1000 filesystems? The pagecache write back
memory consumption on these systems (probably equipped with 10s GB of
RAM) would dwarfs any journal buffers, no?

> On small logs and filesystems, 256kB iclogs doesn't provide any real
> benefit because throughput is limited by log tail pushing (metadata
> writeback), not async transaction throughput.
>
> It's not uncommon for modern disks to have best throughput and/or
> lowest latency at IO sizes of 128kB or smaller.
>
> If you have lots of NVRAM in front of your spinning disks, then log
> IO sizes mostly don't matter - they end up bandwidth limited before
> the iclog size is an issue.

Yes, this matches my observation.

> Testing on a pristine filesystem doesn't show what happens as the
> filesystem ages over years of constant use, and so what provides
> "best performance on empty filesystem" often doesn't provide best
> long term production performance.
>
> And so on.
>
> Storage is complex, filesystems are complex, and no one setting is
> right for everyone. The defaults are intended to be "good enough" in
> the majority of typical user configs.

Yep.

>
> For you're specific storage setup, yes.
>
>> If you, do you suggest to always set logbsize
>> to the maximum supported value?
>
> No. I recommend that people use the defaults, and only if there are
> performance issues with their -actual production workload- should
> they consider changing anything.
>
> Benchmarks rarely match the behaviour of production workloads -
> tuning for benchmarks can actively harm production performance,
> especially over the long term...
>
> Cheers,
>
> Dave.

Ok, very clear.
Thank you so much.