[Cluster-devel] [PATCH 2/2] fsck.gfs2: Improve read ahead in pass1_process_bitmap

[Andrew Price <anprice@redhat.com>]

The read ahead strategy in this function was pretty naive, it just read
in all of the inode blocks, in block-sized reads, before doing any
processing. In my tests with a full filesystem, removing this read ahead
actually sped up the tests, presumably due to issuing a large number of
small I/Os and not keeping them cached. Moving the read ahead into the
processing loop and reading ahead by larger chunks after certain periods
gave improved performance. After some experimentation, I found the best
performance was given by reading ahead 100 blocks after every 50 inodes
processed. Timings were:

    Master        Without R.A.  With this patch
    51:03.41      48:07.40      47:13.66
    50:56.45      48:07.42      47:01.04
    51:02.25      48:00.06      47:05.72

I would expect the performance gain to be larger on filesystems with a
higher density of inode blocks.

Signed-off-by: Andrew Price <anprice@redhat.com>
---
 gfs2/fsck/pass1.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/gfs2/fsck/pass1.c b/gfs2/fsck/pass1.c
index c0f2f1e..653be59 100644
--- a/gfs2/fsck/pass1.c
+++ b/gfs2/fsck/pass1.c
@@ -1532,17 +1532,22 @@ static int pass1_process_bitmap(struct gfs2_sbd *sdp, struct rgrp_tree *rgd, uin
 	uint64_t block;
 	struct gfs2_inode *ip;
 	uint8_t q;
+	/* Readahead numbers arrived at by experiment */
+	unsigned rawin = 50;
+	unsigned ralen = 100 * sdp->bsize;
+	unsigned r = 0;
 
-	/* Issue read-ahead for all dinodes in this bitmap */
-	for (i = 0; i < n; i++)
-		posix_fadvise(sdp->device_fd, ibuf[i] * sdp->bsize, sdp->bsize,
-			      POSIX_FADV_WILLNEED);
 	for (i = 0; i < n; i++) {
 		int is_inode;
 		uint32_t check_magic;
 
 		block = ibuf[i];
 
+		if (r++ == rawin) {
+			posix_fadvise(sdp->device_fd, block * sdp->bsize, ralen, POSIX_FADV_WILLNEED);
+			r = 0;
+		}
+
 		/* skip gfs1 rindex indirect blocks */
 		if (sdp->gfs1 && blockfind(&gfs1_rindex_blks, block)) {
 			log_debug(_("Skipping rindex indir block "
-- 
2.4.3