From: Dave Chinner <david@fromorbit.com>
To: Gionatan Danti <g.danti@assyoma.it>
Cc: linux-xfs@vger.kernel.org
Subject: Re: Question about logbsize default value
Date: Sun, 27 Oct 2019 08:59:09 +1100 [thread overview]
Message-ID: <20191026215909.GK4614@dread.disaster.area> (raw)
In-Reply-To: <51fef5c8e58db12a72b693680c2feaa5@assyoma.it>
On Sat, Oct 26, 2019 at 11:54:02AM +0200, Gionatan Danti wrote:
> Il 26-10-2019 01:39 Dave Chinner ha scritto:
> > Again, it's a trade-off.
> >
> > 256kB iclogs mean that a crash can leave an unrecoverable 2MB hole
> > in the journal, while 32kB iclogs means it's only 256kB.
>
> Sure, but a crash will always cause the loss of unsynced data, especially
> when using deferred logging and/or deferred allocation, right?
Yes, but there's a big difference between 2MB and 256KB, especially
if it's a small filesystem (very common) and the log is only ~10MB
in size.
> > 256kB iclogs mean 2MB of memory usage per filesystem, 32kB is only
> > 256kB. We have users with hundreds of individual XFS filesystems
> > mounted on single machines, and so 256kB iclogs is a lot of wasted
> > memory...
>
> Just wondering: 1000 filesystems with 256k logbsize would result in 2 GB of
> memory consumed by journal buffers. Is this considered too much memory for a
> system managing 1000 filesystems? The pagecache write back memory
> consumption on these systems (probably equipped with 10s GB of RAM) would
> dwarfs any journal buffers, no?
Log buffers are static memory footprint. Page cache memory is
dynamic and can be trimmed to nothing when there is memory pressure
However, memory allocated to log buffers is pinned for the life
of the mount, whether that filesystem is busy or not - the memory is
not reclaimable.
THe 8 log buffers of 32kB each is a good trade-off between
minimising memory footprint and maintaining performance over a wide
range of storage and use cases. If that's still too much memory per
filesystem, then the user can compromise on performance by reducing
the number of logbufs. If performance is too slow, then the user can
increase the memory footprint to improve performance.
The default values sit in the middle ground on both axis - enough
logbufs and iclog size for decent performance but with a small
enough memory footprint that dense or resource constrained
installations are possible to deploy without needing any tweaking.
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
next prev parent reply other threads:[~2019-10-26 21:59 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-10-23 9:40 Question about logbsize default value Gionatan Danti
2019-10-24 21:50 ` Dave Chinner
2019-10-25 7:10 ` Gionatan Danti
2019-10-25 23:39 ` Dave Chinner
2019-10-26 9:54 ` Gionatan Danti
2019-10-26 21:59 ` Dave Chinner [this message]
2019-10-27 18:09 ` Gionatan Danti
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191026215909.GK4614@dread.disaster.area \
--to=david@fromorbit.com \
--cc=g.danti@assyoma.it \
--cc=linux-xfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox