Re: [PATCH 2/6] writeback: allow for dirty metadata accounting

[Dave Chinner <david@fromorbit.com>]

On Thu, Nov 09, 2017 at 02:30:57PM -0500, Josef Bacik wrote:
> From: Josef Bacik <jbacik@fb.com>
> 
> Provide a mechanism for file systems to indicate how much dirty metadata they
> are holding.  This introduces a few things
> 
> 1) Zone stats for dirty metadata, which is the same as the NR_FILE_DIRTY.
> 2) WB stat for dirty metadata.  This way we know if we need to try and call into
> the file system to write out metadata.  This could potentially be used in the
> future to make balancing of dirty pages smarter.

Ok, so when you have 64k page size and 4k metadata block size and
you're using kmalloc() to allocate the storage for the metadata,
how do we make use of all this page-based metadata accounting
stuff?

We could use dirty metadata accounting infrastructure in
XFS so the mm/ subsystem can push on dirty metadata before we
get into reclaim situations, but I just can't see how this code
works when raw pages are not used to back the metadata cache.

That is, XFS can use various different sizes of metadata buffers in
the same filesystem. For example, we use sector sized buffers for
static AG metadata, filesystem blocks for per-AG metadata, some
multiple (1-16) of filesystem blocks for inode buffers, and some
multiple (1-128) of filesytem blocks for directory blocks.

This means we have a combination of buffers  we need to account for
that are made up of:
	heap memory when buffer size < page size,
	single pages when buffer size == page size, and
	multiple pages when buffer size > page size.

The default filesystem config on a 4k page machine with 512 byte
sectors will create buffers in all three categories above which
tends to indicate we can't use this new generic infrastructure as
proposeda.

Any thoughts about how we could efficiently support accounting for
variable sized, non-page based metadata with this generic
infrastructure?

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>