public inbox for linux-xfs@vger.kernel.org
 help / color / mirror / Atom feed
* [PATCH 00/10] xfs: discontiguous buffer support a.k.a. die xfs_dabuf die
@ 2012-04-24  6:33 Dave Chinner
  2012-04-24  6:33 ` [PATCH 01/10] xfs: add trace points for log forces Dave Chinner
                   ` (10 more replies)
  0 siblings, 11 replies; 28+ messages in thread
From: Dave Chinner @ 2012-04-24  6:33 UTC (permalink / raw)
  To: xfs

This patch series builds on top the current patch queue I posted
yesterday. This series replaces the struct xfs-dabuf with an xfs_buf
that can serve the same purpose.

Directory buffers may be made up of multiple extents, but are
currently formed by creating individual buffers and then copying the
data out of them into a linear memory region in a dabuf structure.
All dabuf operations then require walking all the underlying buffers
to change the state of the underlying buffers, and once a dabuf is
modified the contents need to be copied back to the underlying
buffers before they are logged.

All of these operations can be done on a normal xfs_buf, but the
normal xfs_buf does not support multiple disk block ranges or doing
multiple disjoint I/Os to read or write a buffer. Supporting
multiple disk block ranges is not difficult - we simply need to
attach an iovec-like array to the buffer rather than just using a
single block number and length.

Splitting the buffer up into multiple IOs for read and write is not
difficult, either. We already track the number of IO remaining to
complete an IO, so this can be used to wait for the multiple IO
dispatched to complete (for both read and write).

The only interesting twist to this is logging the changes. We can
treat the discontiguous buffer as a single buffer for most purposes
except for formatting the changes into the log. When formatting, we
need to split the changes into a format item per underlying region
so that recovery does not need to know about compound buffers and
can recover each segment of a directory block indivdually as it does
now. The fact that recovery will replay all or none of the
transaction ensures this process is still atomic from a change
recovery point of view.

Further, even though log recovery doesn't use discontiguous buffers,
there will be no confusion between a short buffer written by
recovery and a discontiguous buffer read by the directory code after
mount because the lengths of the buffer will be different. hence we
need no changes to mount or log recovery processing as we already
ensure that all log recovery changes hve been written to disk before
we finish the mount process.

The reason for making this changes is that we can now use a buffer
cache callback to do all the metadata CRC calculations and
verifications across both contiguous and discontiguous directory
blocks. It greatly simplifies the implementation of this code and
makes it consistent with all other metadata buffers. It should also
provide a performance improvement because it avoids double copying
and reduces the number of cached buffers.

I've tested this on 4k/4k (FSB/DB sizes), 512b/64k, and 4k/64k
combinations with xfstests and some dbench, fsmark and compilebench
stress loads. More testing is welcome....

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 28+ messages in thread

end of thread, other threads:[~2012-05-30 19:43 UTC | newest]

Thread overview: 28+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-04-24  6:33 [PATCH 00/10] xfs: discontiguous buffer support a.k.a. die xfs_dabuf die Dave Chinner
2012-04-24  6:33 ` [PATCH 01/10] xfs: add trace points for log forces Dave Chinner
2012-04-30 19:25   ` Mark Tinguely
2012-04-24  6:33 ` [PATCH 02/10] xfs: separate buffer indexing from block map Dave Chinner
2012-04-30 19:28   ` Mark Tinguely
2012-04-30 23:24     ` Dave Chinner
2012-05-01 13:16       ` Mark Tinguely
2012-05-02  1:16         ` Dave Chinner
2012-04-24  6:33 ` [PATCH 03/10] xfs: convert internal buffer functions to pass maps Dave Chinner
2012-05-01 15:13   ` Mark Tinguely
2012-04-24  6:33 ` [PATCH 04/10] xfs: add discontiguous buffer map interface Dave Chinner
2012-05-01 18:10   ` Mark Tinguely
2012-04-24  6:33 ` [PATCH 05/10] xfs: add discontiguous buffer support to transactions Dave Chinner
2012-05-03 19:41   ` Mark Tinguely
2012-04-24  6:33 ` [PATCH 06/10] xfs: struct xfs_buf_log_format isn't variable sized Dave Chinner
2012-05-02 13:39   ` Mark Tinguely
2012-04-24  6:33 ` [PATCH 07/10] xfs: support discontiguous buffers in the xfs_buf_log_item Dave Chinner
2012-05-03 19:42   ` Mark Tinguely
2012-04-24  6:33 ` [PATCH 08/10] xfs: use multiple irec xfs buf support in dabuf Dave Chinner
2012-05-03 19:43   ` Mark Tinguely
2012-04-24  6:33 ` [PATCH 09/10] xfs: remove struct xfs_dabuf and infrastructure Dave Chinner
2012-05-04 19:54   ` Mark Tinguely
2012-04-24  6:33 ` [PATCH 10/10] xfs: factor buffer reading from xfs_dir2_leaf_getdents Dave Chinner
2012-05-04 12:42   ` Mark Tinguely
2012-05-16 17:40 ` [PATCH 00/10] xfs: discontiguous buffer support a.k.a. die xfs_dabuf die Ben Myers
2012-05-23  9:27   ` Dave Chinner
2012-05-30 14:48     ` Ben Myers
2012-05-30 19:48       ` Ben Myers

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox