* [Cluster-devel] [GFS2 PATCH][TRY #2] GFS2: Speed up gfs2_rbm_from_block
[not found] <50432770.21327477.1350648932378.JavaMail.root@redhat.com>
@ 2012-10-19 12:32 ` Bob Peterson
2012-10-19 13:17 ` Steven Whitehouse
0 siblings, 1 reply; 2+ messages in thread
From: Bob Peterson @ 2012-10-19 12:32 UTC (permalink / raw)
To: cluster-devel.redhat.com
Hi,
This is my second attempt at this patch, following Steve's suggestions.
This patch is a rewrite of function gfs2_rbm_from_block. Rather than
looping to find the right bitmap, the code now does a few simple
math calculations.
I compared the performance of both algorithms side by side and the new
algorithm is noticeably faster. Sample instrumentation output from a
"fast" machine:
5 million calls: millisec spent: Orig: 166 New: 113
5 million calls: millisec spent: Orig: 189 New: 114
In addition, I ran postmark (on a somewhat slowr CPU) before the after
the new algorithm was put in place and postmark showed a decent
improvement:
Before the new algorithm:
-------------------------
Time:
645 seconds total
584 seconds of transactions (171 per second)
Files:
150087 created (232 per second)
Creation alone: 100000 files (2083 per second)
Mixed with transactions: 50087 files (85 per second)
49995 read (85 per second)
49991 appended (85 per second)
150087 deleted (232 per second)
Deletion alone: 100174 files (7705 per second)
Mixed with transactions: 49913 files (85 per second)
Data:
273.42 megabytes read (434.08 kilobytes per second)
852.13 megabytes written (1.32 megabytes per second)
With the new algorithm:
-----------------------
Time:
599 seconds total
530 seconds of transactions (188 per second)
Files:
150087 created (250 per second)
Creation alone: 100000 files (1886 per second)
Mixed with transactions: 50087 files (94 per second)
49995 read (94 per second)
49991 appended (94 per second)
150087 deleted (250 per second)
Deletion alone: 100174 files (6260 per second)
Mixed with transactions: 49913 files (94 per second)
Data:
273.42 megabytes read (467.42 kilobytes per second)
852.13 megabytes written (1.42 megabytes per second)
Regards,
Bob Peterson
Red Hat File Systems
Signed-off-by: Bob Peterson <rpeterso@redhat.com>
---
commit f7abb1edc9e61eb5ff5ae9fa1a3964b10f3aa553
Author: Bob Peterson <rpeterso@redhat.com>
Date: Wed Oct 17 09:59:43 2012 -0500
GFS2: Speed up gfs2_rbm_from_block
This patch is a rewrite of function gfs2_rbm_from_block. Rather than
looping to find the right bitmap, the code now does a few simple
math calculations.
diff --git a/fs/gfs2/incore.h b/fs/gfs2/incore.h
index 3d469d3..24bb0b8 100644
--- a/fs/gfs2/incore.h
+++ b/fs/gfs2/incore.h
@@ -621,6 +621,7 @@ struct gfs2_sbd {
u32 sd_hash_bsize_shift;
u32 sd_hash_ptrs; /* Number of pointers in a hash block */
u32 sd_qc_per_block;
+ u32 sd_blocks_per_bitmap;
u32 sd_max_dirres; /* Max blocks needed to add a directory entry */
u32 sd_max_height; /* Max height of a file's metadata tree */
u64 sd_heightsize[GFS2_MAX_META_HEIGHT + 1];
diff --git a/fs/gfs2/ops_fstype.c b/fs/gfs2/ops_fstype.c
index e443966..0e3554e 100644
--- a/fs/gfs2/ops_fstype.c
+++ b/fs/gfs2/ops_fstype.c
@@ -278,6 +278,9 @@ static int gfs2_read_sb(struct gfs2_sbd *sdp, int silent)
sdp->sd_qc_per_block = (sdp->sd_sb.sb_bsize -
sizeof(struct gfs2_meta_header)) /
sizeof(struct gfs2_quota_change);
+ sdp->sd_blocks_per_bitmap = (sdp->sd_sb.sb_bsize -
+ sizeof(struct gfs2_meta_header))
+ * GFS2_NBBY; /* not the rgrp bitmap, subsequent bitmaps only */
/* Compute maximum reservation required to add a entry to a directory */
diff --git a/fs/gfs2/rgrp.c b/fs/gfs2/rgrp.c
index 38fe18f..669b89b 100644
--- a/fs/gfs2/rgrp.c
+++ b/fs/gfs2/rgrp.c
@@ -251,22 +251,25 @@ static u32 gfs2_bitfit(const u8 *buf, const unsigned int len,
static int gfs2_rbm_from_block(struct gfs2_rbm *rbm, u64 block)
{
u64 rblock = block - rbm->rgd->rd_data0;
- u32 goal = (u32)rblock;
- int x;
+ u32 x;
if (WARN_ON_ONCE(rblock > UINT_MAX))
return -EINVAL;
if (block >= rbm->rgd->rd_data0 + rbm->rgd->rd_data)
return -E2BIG;
- for (x = 0; x < rbm->rgd->rd_length; x++) {
- rbm->bi = rbm->rgd->rd_bits + x;
- if (goal < (rbm->bi->bi_start + rbm->bi->bi_len) * GFS2_NBBY) {
- rbm->offset = goal - (rbm->bi->bi_start * GFS2_NBBY);
- break;
- }
- }
+ rbm->bi = rbm->rgd->rd_bits;
+ rbm->offset = (u32)(rblock);
+ /* Check if the block is within the first block */
+ if (rbm->offset < (rbm->bi->bi_start + rbm->bi->bi_len) * GFS2_NBBY)
+ return 0;
+ /* Adjust for the size diff between gfs2_meta_header and gfs2_rgrp */
+ rbm->offset += (sizeof(struct gfs2_rgrp) -
+ sizeof(struct gfs2_meta_header)) * GFS2_NBBY;
+ x = rbm->offset / rbm->rgd->rd_sbd->sd_blocks_per_bitmap;
+ rbm->offset -= x * rbm->rgd->rd_sbd->sd_blocks_per_bitmap;
+ rbm->bi += x;
return 0;
}
^ permalink raw reply related [flat|nested] 2+ messages in thread
* [Cluster-devel] [GFS2 PATCH][TRY #2] GFS2: Speed up gfs2_rbm_from_block
2012-10-19 12:32 ` [Cluster-devel] [GFS2 PATCH][TRY #2] GFS2: Speed up gfs2_rbm_from_block Bob Peterson
@ 2012-10-19 13:17 ` Steven Whitehouse
0 siblings, 0 replies; 2+ messages in thread
From: Steven Whitehouse @ 2012-10-19 13:17 UTC (permalink / raw)
To: cluster-devel.redhat.com
Hi,
That looks good, and good to see that it makes a difference to at least
one workload, as well as the reduction in time. Now in the -nmw git
tree. Thanks,
Steve.
On Fri, 2012-10-19 at 08:32 -0400, Bob Peterson wrote:
> Hi,
>
> This is my second attempt at this patch, following Steve's suggestions.
>
> This patch is a rewrite of function gfs2_rbm_from_block. Rather than
> looping to find the right bitmap, the code now does a few simple
> math calculations.
>
> I compared the performance of both algorithms side by side and the new
> algorithm is noticeably faster. Sample instrumentation output from a
> "fast" machine:
>
> 5 million calls: millisec spent: Orig: 166 New: 113
> 5 million calls: millisec spent: Orig: 189 New: 114
>
> In addition, I ran postmark (on a somewhat slowr CPU) before the after
> the new algorithm was put in place and postmark showed a decent
> improvement:
>
> Before the new algorithm:
> -------------------------
> Time:
> 645 seconds total
> 584 seconds of transactions (171 per second)
>
> Files:
> 150087 created (232 per second)
> Creation alone: 100000 files (2083 per second)
> Mixed with transactions: 50087 files (85 per second)
> 49995 read (85 per second)
> 49991 appended (85 per second)
> 150087 deleted (232 per second)
> Deletion alone: 100174 files (7705 per second)
> Mixed with transactions: 49913 files (85 per second)
>
> Data:
> 273.42 megabytes read (434.08 kilobytes per second)
> 852.13 megabytes written (1.32 megabytes per second)
>
> With the new algorithm:
> -----------------------
> Time:
> 599 seconds total
> 530 seconds of transactions (188 per second)
>
> Files:
> 150087 created (250 per second)
> Creation alone: 100000 files (1886 per second)
> Mixed with transactions: 50087 files (94 per second)
> 49995 read (94 per second)
> 49991 appended (94 per second)
> 150087 deleted (250 per second)
> Deletion alone: 100174 files (6260 per second)
> Mixed with transactions: 49913 files (94 per second)
>
> Data:
> 273.42 megabytes read (467.42 kilobytes per second)
> 852.13 megabytes written (1.42 megabytes per second)
>
> Regards,
>
> Bob Peterson
> Red Hat File Systems
>
> Signed-off-by: Bob Peterson <rpeterso@redhat.com>
> ---
> commit f7abb1edc9e61eb5ff5ae9fa1a3964b10f3aa553
> Author: Bob Peterson <rpeterso@redhat.com>
> Date: Wed Oct 17 09:59:43 2012 -0500
>
> GFS2: Speed up gfs2_rbm_from_block
>
> This patch is a rewrite of function gfs2_rbm_from_block. Rather than
> looping to find the right bitmap, the code now does a few simple
> math calculations.
>
> diff --git a/fs/gfs2/incore.h b/fs/gfs2/incore.h
> index 3d469d3..24bb0b8 100644
> --- a/fs/gfs2/incore.h
> +++ b/fs/gfs2/incore.h
> @@ -621,6 +621,7 @@ struct gfs2_sbd {
> u32 sd_hash_bsize_shift;
> u32 sd_hash_ptrs; /* Number of pointers in a hash block */
> u32 sd_qc_per_block;
> + u32 sd_blocks_per_bitmap;
> u32 sd_max_dirres; /* Max blocks needed to add a directory entry */
> u32 sd_max_height; /* Max height of a file's metadata tree */
> u64 sd_heightsize[GFS2_MAX_META_HEIGHT + 1];
> diff --git a/fs/gfs2/ops_fstype.c b/fs/gfs2/ops_fstype.c
> index e443966..0e3554e 100644
> --- a/fs/gfs2/ops_fstype.c
> +++ b/fs/gfs2/ops_fstype.c
> @@ -278,6 +278,9 @@ static int gfs2_read_sb(struct gfs2_sbd *sdp, int silent)
> sdp->sd_qc_per_block = (sdp->sd_sb.sb_bsize -
> sizeof(struct gfs2_meta_header)) /
> sizeof(struct gfs2_quota_change);
> + sdp->sd_blocks_per_bitmap = (sdp->sd_sb.sb_bsize -
> + sizeof(struct gfs2_meta_header))
> + * GFS2_NBBY; /* not the rgrp bitmap, subsequent bitmaps only */
>
> /* Compute maximum reservation required to add a entry to a directory */
>
> diff --git a/fs/gfs2/rgrp.c b/fs/gfs2/rgrp.c
> index 38fe18f..669b89b 100644
> --- a/fs/gfs2/rgrp.c
> +++ b/fs/gfs2/rgrp.c
> @@ -251,22 +251,25 @@ static u32 gfs2_bitfit(const u8 *buf, const unsigned int len,
> static int gfs2_rbm_from_block(struct gfs2_rbm *rbm, u64 block)
> {
> u64 rblock = block - rbm->rgd->rd_data0;
> - u32 goal = (u32)rblock;
> - int x;
> + u32 x;
>
> if (WARN_ON_ONCE(rblock > UINT_MAX))
> return -EINVAL;
> if (block >= rbm->rgd->rd_data0 + rbm->rgd->rd_data)
> return -E2BIG;
>
> - for (x = 0; x < rbm->rgd->rd_length; x++) {
> - rbm->bi = rbm->rgd->rd_bits + x;
> - if (goal < (rbm->bi->bi_start + rbm->bi->bi_len) * GFS2_NBBY) {
> - rbm->offset = goal - (rbm->bi->bi_start * GFS2_NBBY);
> - break;
> - }
> - }
> + rbm->bi = rbm->rgd->rd_bits;
> + rbm->offset = (u32)(rblock);
> + /* Check if the block is within the first block */
> + if (rbm->offset < (rbm->bi->bi_start + rbm->bi->bi_len) * GFS2_NBBY)
> + return 0;
>
> + /* Adjust for the size diff between gfs2_meta_header and gfs2_rgrp */
> + rbm->offset += (sizeof(struct gfs2_rgrp) -
> + sizeof(struct gfs2_meta_header)) * GFS2_NBBY;
> + x = rbm->offset / rbm->rgd->rd_sbd->sd_blocks_per_bitmap;
> + rbm->offset -= x * rbm->rgd->rd_sbd->sd_blocks_per_bitmap;
> + rbm->bi += x;
> return 0;
> }
>
>
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2012-10-19 13:17 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <50432770.21327477.1350648932378.JavaMail.root@redhat.com>
2012-10-19 12:32 ` [Cluster-devel] [GFS2 PATCH][TRY #2] GFS2: Speed up gfs2_rbm_from_block Bob Peterson
2012-10-19 13:17 ` Steven Whitehouse
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).